-- Logs begin at Mon 2022-10-10 11:12:54 UTC, end at Mon 2022-10-10 12:32:14 UTC. -- Oct 10 11:45:00.269010 dv-cinder-860209-4-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Oct 10 11:45:00.276930 dv-cinder-860209-4-iscsi devstack@keystone.service[81662]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Oct 10 11:45:00.279614 dv-cinder-860209-4-iscsi devstack@keystone.service[81662]: *** Starting uWSGI 2.0.18-debian (64bit) on [Mon Oct 10 11:45:00 2022] *** Oct 10 11:45:00.279614 dv-cinder-860209-4-iscsi devstack@keystone.service[81662]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Oct 10 11:45:00.279614 dv-cinder-860209-4-iscsi devstack@keystone.service[81662]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Oct 10 11:45:00.279614 dv-cinder-860209-4-iscsi devstack@keystone.service[81662]: nodename: dv-cinder-860209-4-iscsi Oct 10 11:45:00.279614 dv-cinder-860209-4-iscsi devstack@keystone.service[81662]: machine: x86_64 Oct 10 11:45:00.279614 dv-cinder-860209-4-iscsi devstack@keystone.service[81662]: clock source: unix Oct 10 11:45:00.279614 dv-cinder-860209-4-iscsi devstack@keystone.service[81662]: pcre jit disabled Oct 10 11:45:00.279614 dv-cinder-860209-4-iscsi devstack@keystone.service[81662]: detected number of CPU cores: 4 Oct 10 11:45:00.279614 dv-cinder-860209-4-iscsi devstack@keystone.service[81662]: current working directory: / Oct 10 11:45:00.279614 dv-cinder-860209-4-iscsi devstack@keystone.service[81662]: detected binary path: /usr/bin/uwsgi-core Oct 10 11:45:00.279614 dv-cinder-860209-4-iscsi devstack@keystone.service[81662]: your processes number limit is 47680 Oct 10 11:45:00.280253 dv-cinder-860209-4-iscsi devstack@keystone.service[81662]: your memory page size is 4096 bytes Oct 10 11:45:00.280253 dv-cinder-860209-4-iscsi devstack@keystone.service[81662]: detected max file descriptor number: 2048 Oct 10 11:45:00.280253 dv-cinder-860209-4-iscsi devstack@keystone.service[81662]: lock engine: pthread robust mutexes Oct 10 11:45:00.280253 dv-cinder-860209-4-iscsi devstack@keystone.service[81662]: thunder lock: enabled Oct 10 11:45:00.280253 dv-cinder-860209-4-iscsi devstack@keystone.service[81662]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Oct 10 11:45:00.280253 dv-cinder-860209-4-iscsi devstack@keystone.service[81662]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Oct 10 11:45:00.319904 dv-cinder-860209-4-iscsi devstack@keystone.service[81662]: Python main interpreter initialized at 0x55d8ee3209c0 Oct 10 11:45:00.319904 dv-cinder-860209-4-iscsi devstack@keystone.service[81662]: python threads support enabled Oct 10 11:45:00.319904 dv-cinder-860209-4-iscsi devstack@keystone.service[81662]: your server socket listen backlog is limited to 100 connections Oct 10 11:45:00.319904 dv-cinder-860209-4-iscsi devstack@keystone.service[81662]: your mercy for graceful operations on workers is 90 seconds Oct 10 11:45:00.320286 dv-cinder-860209-4-iscsi devstack@keystone.service[81662]: mapped 403077 bytes (393 KB) for 2 cores Oct 10 11:45:00.320362 dv-cinder-860209-4-iscsi devstack@keystone.service[81662]: *** Operational MODE: preforking *** Oct 10 11:45:00.320611 dv-cinder-860209-4-iscsi devstack@keystone.service[81662]: *** uWSGI is running in multiple interpreter mode *** Oct 10 11:45:00.320611 dv-cinder-860209-4-iscsi devstack@keystone.service[81662]: spawned uWSGI master process (pid: 81662) Oct 10 11:45:00.320819 dv-cinder-860209-4-iscsi systemd[1]: Started Devstack devstack@keystone.service. Oct 10 11:45:00.322086 dv-cinder-860209-4-iscsi devstack@keystone.service[81662]: spawned uWSGI worker 1 (pid: 81676, cores: 1) Oct 10 11:45:00.322402 dv-cinder-860209-4-iscsi devstack@keystone.service[81662]: spawned uWSGI worker 2 (pid: 81677, cores: 1) Oct 10 11:45:00.322553 dv-cinder-860209-4-iscsi devstack@keystone.service[81662]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Oct 10 11:45:04.712808 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 10 11:45:04.712808 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: from cryptography.utils import int_from_bytes Oct 10 11:45:04.713582 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 10 11:45:04.713582 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: from cryptography.utils import int_from_bytes Oct 10 11:45:04.713696 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 10 11:45:04.713696 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: from cryptography.utils import int_from_bytes Oct 10 11:45:04.714104 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 10 11:45:04.714104 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: from cryptography.utils import int_from_bytes Oct 10 11:45:04.764340 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Oct 10 11:45:04.764498 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Oct 10 11:45:04.764641 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] command line args: [] {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Oct 10 11:45:04.764913 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Oct 10 11:45:04.765016 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Oct 10 11:45:04.765100 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] ================================================================================ {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Oct 10 11:45:04.765218 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Oct 10 11:45:04.765335 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] admin_token = **** {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 10 11:45:04.765422 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] command line args: [] {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Oct 10 11:45:04.765584 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] config_dir = [] {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 10 11:45:04.765661 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Oct 10 11:45:04.765794 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 10 11:45:04.765902 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] ================================================================================ {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Oct 10 11:45:04.765984 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] config_source = [] {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 10 11:45:04.766068 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] admin_token = **** {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 10 11:45:04.766144 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] debug = True {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 10 11:45:04.766310 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 10 11:45:04.766598 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] config_dir = [] {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 10 11:45:04.766598 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 10 11:45:04.766756 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 10 11:45:04.766843 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] config_source = [] {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 10 11:45:04.766917 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 10 11:45:04.766980 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] debug = True {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 10 11:45:04.767251 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 10 11:45:04.767374 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 10 11:45:04.767443 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 10 11:45:04.767505 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 10 11:45:04.767567 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 10 11:45:04.767623 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] list_limit = None {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 10 11:45:04.767804 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] log_config_append = None {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 10 11:45:04.767895 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 10 11:45:04.767962 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 10 11:45:04.768020 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 10 11:45:04.768226 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] log_dir = None {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 10 11:45:04.768313 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] list_limit = None {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 10 11:45:04.768372 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] log_file = None {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 10 11:45:04.768437 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] log_config_append = None {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 10 11:45:04.768496 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 10 11:45:04.768656 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 10 11:45:04.768740 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 10 11:45:04.768805 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] log_dir = None {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 10 11:45:04.769002 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] log_file = None {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 10 11:45:04.769129 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 10 11:45:04.769235 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 10 11:45:04.769315 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 10 11:45:04.769388 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 10 11:45:04.769532 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 10 11:45:04.769674 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 10 11:45:04.769938 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 10 11:45:04.770059 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 10 11:45:04.770195 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 10 11:45:04.770317 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 10 11:45:04.770450 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 10 11:45:04.770625 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 10 11:45:04.770847 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 10 11:45:04.771012 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 10 11:45:04.771115 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 10 11:45:04.771220 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 10 11:45:04.771338 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 10 11:45:04.771434 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 10 11:45:04.771533 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 10 11:45:04.771627 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 10 11:45:04.771729 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 10 11:45:04.771829 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 10 11:45:04.771934 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 10 11:45:04.772035 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 10 11:45:04.772129 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 10 11:45:04.772230 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 10 11:45:04.772230 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 10 11:45:04.772410 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.46/identity {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 10 11:45:04.772524 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 10 11:45:04.772656 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] publish_errors = False {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 10 11:45:04.772770 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 10 11:45:04.772920 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.46/identity {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 10 11:45:04.773048 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 10 11:45:04.773152 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] publish_errors = False {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 10 11:45:04.773244 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 10 11:45:04.773343 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 10 11:45:04.773488 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 10 11:45:04.773590 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 10 11:45:04.773689 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 10 11:45:04.773801 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 10 11:45:04.773901 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] standard_threads = False {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 10 11:45:04.774114 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 10 11:45:04.774229 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 10 11:45:04.774325 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 10 11:45:04.774424 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 10 11:45:04.774424 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 10 11:45:04.774613 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] standard_threads = False {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 10 11:45:04.781779 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] use_journal = False {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 10 11:45:04.781906 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 10 11:45:04.782123 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] use_json = False {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 10 11:45:04.782123 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] use_stderr = False {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 10 11:45:04.782276 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 10 11:45:04.782353 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] use_syslog = False {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 10 11:45:04.782433 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 10 11:45:04.782433 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] use_journal = False {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 10 11:45:04.782559 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 10 11:45:04.782633 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] use_json = False {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 10 11:45:04.782746 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.782827 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] use_stderr = False {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 10 11:45:04.782827 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] use_syslog = False {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 10 11:45:04.782961 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.783029 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 10 11:45:04.783097 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.783178 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.783254 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.783406 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.783485 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.783554 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.783628 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.783702 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.783772 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.783772 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.783944 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.784025 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.784101 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.784165 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.784297 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.784297 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.784428 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.784428 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.784558 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.784558 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.784687 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.784687 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.784860 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.784860 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.784996 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.785069 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.785146 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.785216 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.785284 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.785357 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.785474 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.785547 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.785625 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.785695 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.785815 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.785815 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.785996 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.786071 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.786153 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.786153 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.786334 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.786407 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.786475 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.786542 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.786618 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.786618 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.786779 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.786860 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.786930 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.786998 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.787145 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.787218 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.787292 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.787362 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.787429 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.787493 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.787667 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.787749 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.787825 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.787896 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.787964 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.788032 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.788107 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.788177 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.788246 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.788319 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.788394 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.788463 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.788539 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.788609 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.788677 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.788744 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.788807 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.788870 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.788934 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.788997 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.789055 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.789110 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.789179 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.789244 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.789307 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.789403 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.789464 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.789528 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.789614 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.789683 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.789741 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.789797 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.789852 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.789909 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.789965 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.790028 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.790092 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.790155 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.790219 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.790277 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.790338 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.790406 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.790474 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.790537 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.790602 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.790660 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.790754 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.790835 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.790835 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.790946 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.791009 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.791067 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.791123 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.791179 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.791245 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.791308 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.791371 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.791428 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.791484 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.791540 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.791603 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.791661 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.791724 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.791789 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.791847 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.791909 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.791966 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.792032 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.792090 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.792146 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.792203 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.792259 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.792323 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.792388 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.792451 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.792509 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.792565 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.792620 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.792676 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.792740 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.792804 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.792861 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.792917 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.792972 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.793027 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.793146 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.793253 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.793349 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.793456 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.793552 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.793696 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.793875 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.793974 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.794069 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.794170 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.794265 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.794365 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.794468 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.794570 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.794693 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.794796 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.794891 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.794985 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.795104 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.795220 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.795316 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.795382 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.795480 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.795596 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.795672 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.795739 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.795842 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.795911 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.795985 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.796101 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.796175 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.796250 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.796366 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.796447 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.796568 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.796643 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.796702 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.796852 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.796930 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.797046 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.797121 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.797180 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.797310 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.797401 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.797575 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.797722 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.797853 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.797952 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.798027 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.798110 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.798228 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.798347 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.798454 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.798564 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.798706 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.798847 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.799011 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.799112 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.799185 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.799298 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.799418 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.799547 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.799663 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.799785 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.799908 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.799993 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.800104 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] auth.external = None {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.800212 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.800336 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.800432 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.800643 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.800753 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.800852 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.800933 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.801632 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] auth.password = None {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.801737 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.801864 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] auth.token = None {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.801952 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] auth.external = None {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.802097 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.802194 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.802336 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.802430 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.802585 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.802683 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.802843 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.802931 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] auth.password = None {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.803093 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.803182 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] auth.token = None {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.803335 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.803422 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.803565 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.803650 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.803802 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] credential.caching = True {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.803889 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.804039 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.804124 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.804290 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.804420 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.804544 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.804656 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.804779 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.804861 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.805010 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.805089 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] credential.caching = True {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.805246 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.805328 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.805492 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.805570 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.805744 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.805824 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.805982 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.806068 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.806226 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.806304 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.806467 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.806545 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.806745 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.806830 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.807007 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.807085 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.807249 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.807675 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.807675 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.810709 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.810709 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] federation.caching = True {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.810905 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.810981 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.811053 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.811132 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.811220 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.811315 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.811389 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.811459 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.811529 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.811608 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.811681 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] federation.caching = True {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.811751 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.811821 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.811891 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.811990 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.812076 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.812150 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.812220 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.812290 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.812356 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.812427 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.812505 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.812578 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.812647 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] identity.caching = True {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.812716 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.812792 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.812864 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.812942 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.813014 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.813083 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.813153 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.813222 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.813291 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.813369 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.813482 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] identity.caching = True {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.813559 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.813629 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.813698 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.813768 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.813836 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.813958 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.814034 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.814104 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.814174 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.814244 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.814323 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.814395 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.814464 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.814533 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.814602 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.814695 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.814782 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.814856 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.814926 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.814996 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.815114 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.815199 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.815271 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.815341 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.815410 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.815480 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.815549 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.815626 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.815704 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.815775 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.815852 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.815924 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.815994 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.816072 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.816144 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.816144 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.816269 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.816339 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.816408 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.816485 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.816576 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.817270 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.817361 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.817499 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.817660 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.817751 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.817872 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.817981 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.818055 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.818237 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.818318 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.818481 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.818562 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.818749 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.818834 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.818983 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.819063 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.819223 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.819304 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.819452 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.819531 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.819688 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.819767 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.819924 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.820011 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.820171 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.820251 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.820401 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.820485 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.820645 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.820776 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.820883 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.821032 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.821111 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.821286 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.821374 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.821520 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.821599 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.821752 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.821831 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.822041 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.822178 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.822301 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.822439 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.822559 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.822693 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.822811 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.822940 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.823058 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.823172 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] ldap.user = None {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.823285 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.823398 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.823511 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.823633 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.823748 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.823862 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.824343 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.824483 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.824602 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.824726 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.824842 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.824957 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.825072 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] ldap.user = None {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.825189 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.825304 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.825427 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.825543 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.825658 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.825773 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.825887 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.826009 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.826133 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.826256 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.826362 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.826464 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.826577 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.826707 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.826838 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.826956 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.827071 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.827192 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.827309 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.827432 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.827548 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.827663 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.827778 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.827894 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.828009 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.828133 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.828255 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.828372 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.828487 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.828602 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.828716 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.828840 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.828956 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.829072 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.829186 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.829301 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.829416 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.829545 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.829663 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.829778 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.829893 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.830008 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.830122 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.830249 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.830366 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.830481 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.830596 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.830816 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.830935 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.831127 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.831248 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.831364 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.831479 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.831594 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.831767 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.831895 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.832012 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.832131 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.832273 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.832391 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.832508 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] resource.caching = True {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.832632 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.832749 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.832864 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.832979 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.833095 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.833210 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.833333 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.833551 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.833651 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.833746 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.833841 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.833937 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.834043 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] resource.caching = True {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.834141 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.834238 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.834334 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.834430 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.834525 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.834629 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.834751 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] role.caching = True {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.834850 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.834952 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] role.driver = sql {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.834952 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.835132 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.835132 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.839350 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.839350 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.839574 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.839574 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.839790 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.839921 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.840034 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.840154 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] role.caching = True {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.840267 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.840380 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] role.driver = sql {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.840494 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.840620 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.840730 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.840844 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.840958 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.841072 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.841187 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.841187 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.841409 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.841522 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.841639 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.841751 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.841863 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.841989 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.842100 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.842100 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.842311 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.842421 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.842545 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.842656 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.842806 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.842920 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.843035 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.843154 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.843284 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.843395 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.843507 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.843630 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.843742 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.843855 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.843980 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.844090 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.844203 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.844315 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.844429 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.844542 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.844669 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.844783 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.844899 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.845012 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.845126 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.845238 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.845413 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.845523 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.845641 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.845757 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.845886 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.846000 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.846130 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.846243 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.846367 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.846482 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.846599 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.846781 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.846931 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] token.caching = True {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.847042 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.847156 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.847268 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.847382 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.847493 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.847618 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.847728 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.848320 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.848447 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.848558 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.848669 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.848794 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.848903 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] token.caching = True {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.849014 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.849126 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.849239 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.849361 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.849470 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.849582 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.849694 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.849806 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.849918 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.850046 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.850155 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.850268 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.850380 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.850493 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.850651 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.850809 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.850922 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.851050 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.851174 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.851288 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.851405 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.851520 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.851635 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] database.connection = **** {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.851767 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.851879 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.851995 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.852110 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.852225 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.852340 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.852469 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.852611 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.853093 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.853206 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.853206 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.853429 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.853547 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.853675 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] database.connection = **** {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.853794 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.853911 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.854027 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.854142 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.854259 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.854458 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.854571 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.854748 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.854861 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.854975 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.855129 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.855254 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.855362 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.855523 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.855632 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.855751 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.855862 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.855986 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.856098 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.856211 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.856323 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.856323 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81677) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Oct 10 11:45:04.856535 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.856535 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.856535 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.856535 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.856535 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.856535 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.856535 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.856535 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 10 11:45:04.856535 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81676) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Oct 10 11:45:04.887913 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81676) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 10 11:45:04.888105 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81677) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 10 11:45:04.888192 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81677) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 10 11:45:04.888292 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81676) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 10 11:45:04.888372 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81677) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 10 11:45:04.888448 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81676) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 10 11:45:04.896915 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81677) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 10 11:45:04.897084 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81676) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 10 11:45:04.926543 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81676) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 10 11:45:04.926841 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81676) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 10 11:45:04.927493 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81677) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 10 11:45:04.927722 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81677) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 10 11:45:04.987022 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81677) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 10 11:45:04.987258 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81676) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 10 11:45:05.048485 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81677) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 10 11:45:05.048662 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81677) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 10 11:45:05.048817 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81676) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 10 11:45:05.049100 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81676) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 10 11:45:05.220914 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81677) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 10 11:45:05.221127 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81677) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 10 11:45:05.221200 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81676) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 10 11:45:05.221352 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81677) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 10 11:45:05.221436 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81676) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 10 11:45:05.221575 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81677) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 10 11:45:05.221666 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81676) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 10 11:45:05.221796 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81677) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 10 11:45:05.221877 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81676) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 10 11:45:05.222014 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81677) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 10 11:45:05.222094 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81676) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 10 11:45:05.222233 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81676) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 10 11:45:05.255279 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81677) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 10 11:45:05.255549 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81676) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 10 11:45:05.255549 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81676) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 10 11:45:05.255769 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81677) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 10 11:45:05.255889 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81676) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 10 11:45:05.256039 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81677) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 10 11:45:05.256161 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81676) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 10 11:45:05.256279 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81677) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 10 11:45:05.256399 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81676) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 10 11:45:05.256516 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81677) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 10 11:45:05.256645 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81676) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 10 11:45:05.256767 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81677) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 10 11:45:05.256767 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81677) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 10 11:45:05.256968 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81676) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 10 11:45:05.268257 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81677) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 10 11:45:05.268503 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81676) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 10 11:45:05.282269 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81677) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 10 11:45:05.282519 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81677) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 10 11:45:05.282660 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81676) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 10 11:45:05.282898 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81677) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 10 11:45:05.283017 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81676) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 10 11:45:05.283165 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81677) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 10 11:45:05.283286 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81676) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 10 11:45:05.283409 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81677) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 10 11:45:05.283535 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81676) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 10 11:45:05.283661 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81677) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 10 11:45:05.283797 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81676) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 10 11:45:05.283921 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81677) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 10 11:45:05.284041 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81676) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 10 11:45:05.284041 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81676) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 10 11:45:05.302735 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81677) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:45:05.302735 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81677) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Oct 10 11:45:05.302735 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81677) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:45:05.302735 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81677) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Oct 10 11:45:05.302735 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81677) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:45:05.302735 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81677) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:45:05.303207 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81677) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:45:05.303326 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81677) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:45:05.309434 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81676) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:45:05.309645 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81676) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Oct 10 11:45:05.310135 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81676) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:45:05.310343 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81676) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Oct 10 11:45:05.310626 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81676) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:45:05.310995 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81676) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:45:05.311267 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81676) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:45:05.311534 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81676) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:45:05.313230 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81677) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:45:05.313534 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81677) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:45:05.313813 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81677) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:45:05.314102 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81677) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:45:05.321470 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81677) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 10 11:45:05.321838 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81676) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:45:05.322147 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81676) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:45:05.322428 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81676) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:45:05.322761 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81676) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:45:05.324524 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81677) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 10 11:45:05.324820 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81677) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:45:05.325113 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81677) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:45:05.325405 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81677) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:45:05.325698 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81677) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:45:05.325967 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81677) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:45:05.326249 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81677) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:45:05.326531 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81677) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:45:05.326839 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81677) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:45:05.327120 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81677) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:45:05.327395 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81677) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:45:05.330081 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81676) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 10 11:45:05.333896 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81676) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 10 11:45:05.334209 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81676) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:45:05.334504 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81676) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:45:05.334832 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81676) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:45:05.335125 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81676) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:45:05.335397 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81676) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:45:05.335680 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81676) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:45:05.335963 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81676) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:45:05.336246 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81676) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:45:05.336529 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81676) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:45:05.336805 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81676) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:45:05.351702 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81677) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:45:05.353227 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81677) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 10 11:45:05.353523 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81677) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:45:05.358425 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81677) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 10 11:45:05.358751 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81677) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:45:05.359044 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81677) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:45:05.361065 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81676) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:45:05.362641 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81676) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 10 11:45:05.362989 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81676) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:45:05.365450 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81677) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 10 11:45:05.365741 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81677) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:45:05.367774 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81676) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 10 11:45:05.368058 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81676) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:45:05.368349 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81676) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:45:05.371788 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81677) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 10 11:45:05.372084 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81677) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:45:05.372480 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81677) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:45:05.372758 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81677) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:45:05.373042 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81677) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:45:05.373323 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81677) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:45:05.373599 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81677) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:45:05.373876 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81677) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:45:05.374904 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81676) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 10 11:45:05.375453 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81676) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:45:05.381460 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81676) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 10 11:45:05.381758 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81676) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:45:05.382056 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81676) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:45:05.382348 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81676) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:45:05.382631 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81676) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:45:05.382942 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81676) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:45:05.383221 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81676) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:45:05.383600 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81676) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:45:05.389752 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81677) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:45:05.390035 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81677) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:45:05.395275 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81677) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 10 11:45:05.397696 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81677) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:45:05.397998 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81677) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:45:05.399956 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81676) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:45:05.400246 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81676) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:45:05.402272 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81677) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 10 11:45:05.404241 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81676) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 10 11:45:05.405450 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81677) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 10 11:45:05.407545 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81676) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:45:05.407848 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81676) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:45:05.408816 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81677) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:45:05.409121 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81677) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:45:05.409409 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81677) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:45:05.409696 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81677) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:45:05.410001 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81677) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:45:05.410343 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81677) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:45:05.412789 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81676) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 10 11:45:05.416093 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81676) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 10 11:45:05.419474 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81676) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:45:05.419780 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81676) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:45:05.420074 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81676) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:45:05.420366 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81676) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:45:05.420665 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81676) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:45:05.420957 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81676) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:45:05.426772 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81677) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 10 11:45:05.427066 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81677) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:45:05.427344 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81677) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:45:05.427611 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81677) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:45:05.434448 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81677) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:45:05.436148 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81677) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:45:05.436995 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81676) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 10 11:45:05.437292 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81676) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:45:05.437571 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81676) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:45:05.437690 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81677) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:45:05.437842 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81676) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:45:05.437971 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81677) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:45:05.440696 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81677) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 10 11:45:05.441000 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81677) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:45:05.441287 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81677) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:45:05.441598 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81677) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:45:05.442008 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81677) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:45:05.444866 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81676) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:45:05.446427 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81676) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:45:05.448114 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81676) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:45:05.448399 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81676) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:45:05.451151 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81676) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 10 11:45:05.451445 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81676) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:45:05.451745 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81676) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:45:05.452035 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81676) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:45:05.452677 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81676) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:45:05.453278 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81677) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 10 11:45:05.453577 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81677) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:45:05.453864 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81677) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:45:05.454171 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81677) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:45:05.454532 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81677) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:45:05.454843 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81677) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:45:05.455127 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81677) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:45:05.464501 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81676) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 10 11:45:05.464799 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81676) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:45:05.465090 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81676) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:45:05.465375 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81676) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:45:05.469795 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81676) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:45:05.469795 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81676) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:45:05.469795 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81676) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:45:05.471140 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81677) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 10 11:45:05.474603 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81677) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 10 11:45:05.477550 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81677) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:45:05.479047 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81677) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:45:05.480509 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81677) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 10 11:45:05.480872 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81677) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:45:05.481319 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81677) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:45:05.482625 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81676) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 10 11:45:05.485578 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81676) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 10 11:45:05.487805 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81677) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:45:05.488644 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81676) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:45:05.489301 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81677) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 10 11:45:05.490190 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81676) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:45:05.491740 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81676) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 10 11:45:05.492032 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81676) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:45:05.492198 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81677) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:45:05.492443 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81676) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:45:05.492601 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81677) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:45:05.492892 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81677) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:45:05.493172 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81677) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:45:05.498903 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81676) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:45:05.500641 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81676) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 10 11:45:05.501244 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81677) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 10 11:45:05.501546 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81677) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:45:05.501829 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81677) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:45:05.503773 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81676) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:45:05.504221 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81676) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:45:05.504518 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81676) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:45:05.504805 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81676) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:45:05.508784 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81677) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 10 11:45:05.509074 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81677) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:45:05.509371 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81677) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:45:05.509649 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81677) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:45:05.509934 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81677) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:45:05.515492 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81677) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:45:05.515492 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81677) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:45:05.515492 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81677) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:45:05.515492 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81677) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:45:05.515492 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81677) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:45:05.515492 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81677) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:45:05.515851 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81677) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:45:05.515851 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81677) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:45:05.515851 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81677) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:45:05.516037 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81676) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 10 11:45:05.516037 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81676) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:45:05.516037 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81676) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:45:05.520791 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81676) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 10 11:45:05.521077 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81676) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:45:05.521366 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81676) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:45:05.521650 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81676) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:45:05.521933 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81676) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:45:05.522211 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81676) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:45:05.522493 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81676) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:45:05.522811 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81676) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:45:05.523109 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81676) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:45:05.523387 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81676) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:45:05.523673 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81676) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:45:05.523941 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81676) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:45:05.524212 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81676) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:45:05.524487 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81676) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 10 11:45:05.547860 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x55d8ee3209c0 pid: 81677 (default app) Oct 10 11:45:05.551319 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2dfcc1-2d53-4af6-a439-d872b29d37df None None] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:05.551613 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2dfcc1-2d53-4af6-a439-d872b29d37df None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:05.551891 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2dfcc1-2d53-4af6-a439-d872b29d37df None None] PATH_INFO: `/v3` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:05.556187 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1/1] 10.222.0.46 () {50 vars in 791 bytes} [Mon Oct 10 11:45:05 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:45:05.560135 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x55d8ee3209c0 pid: 81676 (default app) Oct 10 11:45:13.287535 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0142a55d-3daf-413b-9738-542ce4b17d21 None None] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:13.288065 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0142a55d-3daf-413b-9738-542ce4b17d21 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:13.288065 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0142a55d-3daf-413b-9738-542ce4b17d21 None None] PATH_INFO: `/` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:13.289399 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 2/2] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 10 11:45:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 10 11:45:13.296957 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37a8120-f588-4bb1-bf97-c80b4f0cee79 None None] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:13.297410 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37a8120-f588-4bb1-bf97-c80b4f0cee79 None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:13.297514 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37a8120-f588-4bb1-bf97-c80b4f0cee79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:13.368187 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: INFO dbcounter [None req-f37a8120-f588-4bb1-bf97-c80b4f0cee79 None None] Registered counter for database keystone Oct 10 11:45:13.388246 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG oslo_db.sqlalchemy.engines [None req-f37a8120-f588-4bb1-bf97-c80b4f0cee79 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81676) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Oct 10 11:45:13.395150 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writer thread running {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 10 11:45:13.571165 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG passlib.handlers.bcrypt [None req-f37a8120-f588-4bb1-bf97-c80b4f0cee79 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81676) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Oct 10 11:45:13.571476 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG passlib.handlers.bcrypt [None req-f37a8120-f588-4bb1-bf97-c80b4f0cee79 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81676) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Oct 10 11:45:13.630007 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.auth.core [None req-f37a8120-f588-4bb1-bf97-c80b4f0cee79 None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:45:13.631707 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-f37a8120-f588-4bb1-bf97-c80b4f0cee79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:45:13.757743 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1/3] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 10 11:45:13 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 464 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Oct 10 11:45:14.221338 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0142a55d-3daf-413b-9738-542ce4b17d21 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:14.291830 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: INFO dbcounter [None req-0142a55d-3daf-413b-9738-542ce4b17d21 None None] Registered counter for database keystone Oct 10 11:45:14.313145 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG oslo_db.sqlalchemy.engines [None req-0142a55d-3daf-413b-9738-542ce4b17d21 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81677) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Oct 10 11:45:14.319435 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writer thread running {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 10 11:45:14.438617 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0142a55d-3daf-413b-9738-542ce4b17d21 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:14.443658 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5758196d-8403-42f3-bbef-a5af1321ba4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:14.445644 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5758196d-8403-42f3-bbef-a5af1321ba4c admin admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:14.445644 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5758196d-8403-42f3-bbef-a5af1321ba4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:14.445644 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5758196d-8403-42f3-bbef-a5af1321ba4c admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:14.453794 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5758196d-8403-42f3-bbef-a5af1321ba4c admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:14.506014 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: WARNING keystone.common.rbac_enforcer.enforcer [None req-5758196d-8403-42f3-bbef-a5af1321ba4c admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Oct 10 11:45:14.506354 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5758196d-8403-42f3-bbef-a5af1321ba4c admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:14.516448 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 3/4] 10.222.0.46 () {58 vars in 1324 bytes} [Mon Oct 10 11:45:14 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 10 11:45:14.523069 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37a8120-f588-4bb1-bf97-c80b4f0cee79 None None] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:14.540414 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37a8120-f588-4bb1-bf97-c80b4f0cee79 None None] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:14.545046 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd3282f-c156-4a21-bcc4-b0870889ca75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:14.546286 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd3282f-c156-4a21-bcc4-b0870889ca75 admin admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:14.546575 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd3282f-c156-4a21-bcc4-b0870889ca75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:14.546891 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd3282f-c156-4a21-bcc4-b0870889ca75 admin admin] PATH_INFO: `/v3/projects/ec187f354d154dc98e045921aa4ca7fd` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:14.548410 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd3282f-c156-4a21-bcc4-b0870889ca75 admin admin] RBAC: Authorizing `identity:get_project(project_id=ec187f354d154dc98e045921aa4ca7fd)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:14.600874 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: WARNING keystone.common.rbac_enforcer.enforcer [None req-0dd3282f-c156-4a21-bcc4-b0870889ca75 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Oct 10 11:45:14.601225 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd3282f-c156-4a21-bcc4-b0870889ca75 admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:14.605769 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 2/5] 10.222.0.46 () {56 vars in 1207 bytes} [Mon Oct 10 11:45:14 2022] GET /identity/v3/projects/ec187f354d154dc98e045921aa4ca7fd => generated 344 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 11:45:16.773943 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d948460-6619-4c06-8020-8c78bbc6377d None None] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:16.774424 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d948460-6619-4c06-8020-8c78bbc6377d None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:16.774424 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d948460-6619-4c06-8020-8c78bbc6377d None None] PATH_INFO: `/` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:16.778470 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 4/6] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 10 11:45:16 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 10 11:45:16.784026 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24d2626-3782-40a9-a2ed-60ae6845b9e4 None None] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:16.784456 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24d2626-3782-40a9-a2ed-60ae6845b9e4 None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:16.784632 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24d2626-3782-40a9-a2ed-60ae6845b9e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:16.805510 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.auth.core [None req-c24d2626-3782-40a9-a2ed-60ae6845b9e4 None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:45:16.807060 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-c24d2626-3782-40a9-a2ed-60ae6845b9e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:45:16.815083 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 3/7] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 10 11:45:16 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Oct 10 11:45:17.079717 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d948460-6619-4c06-8020-8c78bbc6377d None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:17.091567 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d948460-6619-4c06-8020-8c78bbc6377d None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:17.095752 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527d8a1b-9fed-401c-8d79-bbb5d7398381 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:17.096343 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527d8a1b-9fed-401c-8d79-bbb5d7398381 admin admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:17.096618 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527d8a1b-9fed-401c-8d79-bbb5d7398381 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:17.096888 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527d8a1b-9fed-401c-8d79-bbb5d7398381 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:17.105029 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527d8a1b-9fed-401c-8d79-bbb5d7398381 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:17.106810 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527d8a1b-9fed-401c-8d79-bbb5d7398381 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:17.115913 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 5/8] 10.222.0.46 () {58 vars in 1324 bytes} [Mon Oct 10 11:45:17 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Oct 10 11:45:17.121237 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24d2626-3782-40a9-a2ed-60ae6845b9e4 None None] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:17.130757 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24d2626-3782-40a9-a2ed-60ae6845b9e4 None None] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:17.135133 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735e7b4a-ec49-4c45-ac85-66663b95d344 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:17.135755 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735e7b4a-ec49-4c45-ac85-66663b95d344 admin admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:17.136037 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735e7b4a-ec49-4c45-ac85-66663b95d344 admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:17.136313 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735e7b4a-ec49-4c45-ac85-66663b95d344 admin admin] PATH_INFO: `/v3/users/8cbbc9233ced43d487079e9479dc555c` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:17.137662 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735e7b4a-ec49-4c45-ac85-66663b95d344 admin admin] RBAC: Authorizing `identity:get_user(user_id=8cbbc9233ced43d487079e9479dc555c)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:17.139518 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735e7b4a-ec49-4c45-ac85-66663b95d344 admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:17.141045 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 4/9] 10.222.0.46 () {56 vars in 1198 bytes} [Mon Oct 10 11:45:17 2022] GET /identity/v3/users/8cbbc9233ced43d487079e9479dc555c => generated 247 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:45:19.329849 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19ecb1d-a6af-438a-8f6a-a0a70f264bd6 None None] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:19.330432 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19ecb1d-a6af-438a-8f6a-a0a70f264bd6 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:19.330432 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19ecb1d-a6af-438a-8f6a-a0a70f264bd6 None None] PATH_INFO: `/` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:19.331784 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 6/10] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 10 11:45:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:45:19.337702 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e44cd7-4b65-4161-bbcd-4dc0fdf1a156 None None] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:19.338146 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e44cd7-4b65-4161-bbcd-4dc0fdf1a156 None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:19.338325 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e44cd7-4b65-4161-bbcd-4dc0fdf1a156 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:19.356089 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.auth.core [None req-29e44cd7-4b65-4161-bbcd-4dc0fdf1a156 None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:45:19.357270 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-29e44cd7-4b65-4161-bbcd-4dc0fdf1a156 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:45:19.421253 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969b190b-1e84-442a-a863-6bc34b7f5627 None None] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:19.421654 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969b190b-1e84-442a-a863-6bc34b7f5627 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:19.422002 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969b190b-1e84-442a-a863-6bc34b7f5627 None None] PATH_INFO: `/` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:19.423477 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 7/11] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 10 11:45:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 10 11:45:19.428890 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1171b6-23b6-429f-b65c-8132c4bc4e9b None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:19.429264 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1171b6-23b6-429f-b65c-8132c4bc4e9b None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:19.429545 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1171b6-23b6-429f-b65c-8132c4bc4e9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:19.439348 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 5/12] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 10 11:45:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 10 11:45:19.474003 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG passlib.handlers.bcrypt [None req-2e1171b6-23b6-429f-b65c-8132c4bc4e9b None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81677) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Oct 10 11:45:19.474393 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG passlib.handlers.bcrypt [None req-2e1171b6-23b6-429f-b65c-8132c4bc4e9b None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81677) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Oct 10 11:45:19.523574 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-2e1171b6-23b6-429f-b65c-8132c4bc4e9b None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:45:19.525149 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-2e1171b6-23b6-429f-b65c-8132c4bc4e9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:45:19.623503 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 8/13] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 10 11:45:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 10 11:45:19.703774 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e44cd7-4b65-4161-bbcd-4dc0fdf1a156 None None] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:19.729342 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:19.809304 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e44cd7-4b65-4161-bbcd-4dc0fdf1a156 None None] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:19.842970 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4afd77b-8de0-45b5-bf0a-a5d15c0d628b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:19.843617 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4afd77b-8de0-45b5-bf0a-a5d15c0d628b None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:19.843902 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4afd77b-8de0-45b5-bf0a-a5d15c0d628b None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:19.844203 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4afd77b-8de0-45b5-bf0a-a5d15c0d628b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:19.858113 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4afd77b-8de0-45b5-bf0a-a5d15c0d628b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 10 11:45:19.867811 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application [None req-f4afd77b-8de0-45b5-bf0a-a5d15c0d628b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 10 11:45:19.867811 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:45:19.867811 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:45:19.867811 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:45:19.867811 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:45:19.867811 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:45:19.867811 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:45:19.867811 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:45:19.867811 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:45:19.867811 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:45:19.867811 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:45:19.867811 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:45:19.867811 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 10 11:45:19.867811 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 10 11:45:19.867811 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 10 11:45:19.867811 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 10 11:45:19.868792 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 10 11:45:19.868792 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 10 11:45:19.868792 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 10 11:45:19.868792 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 10 11:45:19.868792 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:45:19.868792 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:45:19.868792 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:45:19.868792 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:45:19.868792 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:45:19.868792 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:45:19.868792 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:45:19.868792 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application with Lock( Oct 10 11:45:19.868792 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:45:19.868792 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:45:19.868792 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:45:19.868792 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:45:19.869699 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:45:19.869699 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:45:19.869699 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:45:19.869699 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:45:19.869699 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 10 11:45:19.869699 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 10 11:45:19.869699 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 10 11:45:19.869699 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 10 11:45:19.869699 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 10 11:45:19.869699 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 10 11:45:19.869699 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 10 11:45:19.869699 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Oct 10 11:45:19.869699 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 6/14] 10.222.0.46 () {56 vars in 1096 bytes} [Mon Oct 10 11:45:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:45:19.875117 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1171b6-23b6-429f-b65c-8132c4bc4e9b None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:19.895749 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4afd77b-8de0-45b5-bf0a-a5d15c0d628b None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:19.976580 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1171b6-23b6-429f-b65c-8132c4bc4e9b None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:19.978958 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:19.979666 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:19.997447 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4afd77b-8de0-45b5-bf0a-a5d15c0d628b None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:20.013767 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2ea3ff-5c57-4dde-8c24-51d1f17ad5b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:20.014635 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2ea3ff-5c57-4dde-8c24-51d1f17ad5b0 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:20.014945 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2ea3ff-5c57-4dde-8c24-51d1f17ad5b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:20.015227 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2ea3ff-5c57-4dde-8c24-51d1f17ad5b0 None admin] PATH_INFO: `/v3/roles` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:20.016091 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2ea3ff-5c57-4dde-8c24-51d1f17ad5b0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:45:20.016395 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2ea3ff-5c57-4dde-8c24-51d1f17ad5b0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:20.017862 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2ea3ff-5c57-4dde-8c24-51d1f17ad5b0 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:20.030830 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 9/15] 10.222.0.46 () {56 vars in 1099 bytes} [Mon Oct 10 11:45:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 11:45:20.036819 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2ea3ff-5c57-4dde-8c24-51d1f17ad5b0 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:20.037312 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea056174-5ab7-4980-9e9e-fc10abf4a3b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:20.037942 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea056174-5ab7-4980-9e9e-fc10abf4a3b8 None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:20.038225 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea056174-5ab7-4980-9e9e-fc10abf4a3b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:20.038500 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea056174-5ab7-4980-9e9e-fc10abf4a3b8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:20.106626 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea056174-5ab7-4980-9e9e-fc10abf4a3b8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:20.108144 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea056174-5ab7-4980-9e9e-fc10abf4a3b8 None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:20.140277 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2ea3ff-5c57-4dde-8c24-51d1f17ad5b0 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:20.173840 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e0def7-c1ae-4422-ad97-e0538d01ed28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:20.174718 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e0def7-c1ae-4422-ad97-e0538d01ed28 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:20.175016 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e0def7-c1ae-4422-ad97-e0538d01ed28 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:20.175274 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e0def7-c1ae-4422-ad97-e0538d01ed28 None admin] PATH_INFO: `/v3/users/8cbbc9233ced43d487079e9479dc555c` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:20.176797 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e0def7-c1ae-4422-ad97-e0538d01ed28 None admin] RBAC: Authorizing `identity:get_user(user_id=8cbbc9233ced43d487079e9479dc555c)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:20.178286 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e0def7-c1ae-4422-ad97-e0538d01ed28 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:20.179900 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 10/16] 10.222.0.46 () {56 vars in 1177 bytes} [Mon Oct 10 11:45:20 2022] GET /identity/v3/users/8cbbc9233ced43d487079e9479dc555c => generated 247 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 11:45:20.185499 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e0def7-c1ae-4422-ad97-e0538d01ed28 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:20.192328 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:20.211395 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 7/17] 10.222.0.46 () {58 vars in 1282 bytes} [Mon Oct 10 11:45:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Oct 10 11:45:20.217068 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea056174-5ab7-4980-9e9e-fc10abf4a3b8 None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:20.223164 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:20.289685 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e0def7-c1ae-4422-ad97-e0538d01ed28 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:20.317397 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea056174-5ab7-4980-9e9e-fc10abf4a3b8 None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:20.324275 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08acfdb-135e-480c-bd7e-c4cd8400b9a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:20.324911 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08acfdb-135e-480c-bd7e-c4cd8400b9a9 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:20.325194 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08acfdb-135e-480c-bd7e-c4cd8400b9a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:20.325471 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08acfdb-135e-480c-bd7e-c4cd8400b9a9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:20.326591 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08acfdb-135e-480c-bd7e-c4cd8400b9a9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:20.328243 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08acfdb-135e-480c-bd7e-c4cd8400b9a9 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:20.331318 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 11/18] 10.222.0.46 () {56 vars in 1108 bytes} [Mon Oct 10 11:45:20 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 11:45:20.336890 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08acfdb-135e-480c-bd7e-c4cd8400b9a9 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:20.354454 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ccf4b4-7ca4-4f8f-beb5-f87d27f3a0e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:20.355235 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ccf4b4-7ca4-4f8f-beb5-f87d27f3a0e6 None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:20.355572 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ccf4b4-7ca4-4f8f-beb5-f87d27f3a0e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:20.355886 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ccf4b4-7ca4-4f8f-beb5-f87d27f3a0e6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:20.368783 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ccf4b4-7ca4-4f8f-beb5-f87d27f3a0e6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:20.370434 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ccf4b4-7ca4-4f8f-beb5-f87d27f3a0e6 None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:20.388834 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application [None req-43ccf4b4-7ca4-4f8f-beb5-f87d27f3a0e6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:45:20.388834 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:45:20.388834 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:45:20.388834 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:45:20.388834 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:45:20.388834 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:45:20.388834 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:45:20.388834 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:45:20.388834 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:45:20.388834 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:45:20.388834 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:45:20.388834 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:45:20.388834 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 10 11:45:20.388834 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 10 11:45:20.388834 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 10 11:45:20.388834 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 10 11:45:20.389849 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:45:20.389849 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:45:20.389849 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:45:20.389849 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:45:20.389849 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:45:20.389849 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:45:20.389849 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:45:20.389849 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application with Lock( Oct 10 11:45:20.389849 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:45:20.389849 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:45:20.389849 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:45:20.389849 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:45:20.389849 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:45:20.389849 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:45:20.389849 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:45:20.389849 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:45:20.389849 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 10 11:45:20.390708 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 10 11:45:20.390708 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:45:20.390708 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Oct 10 11:45:20.390708 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 8/19] 10.222.0.46 () {56 vars in 1108 bytes} [Mon Oct 10 11:45:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 10 11:45:20.396765 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ccf4b4-7ca4-4f8f-beb5-f87d27f3a0e6 None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:20.432896 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:20.441439 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08acfdb-135e-480c-bd7e-c4cd8400b9a9 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:20.442999 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:20.477561 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1aa50ca-4f1b-4970-8db1-291377d7d3b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:20.478176 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1aa50ca-4f1b-4970-8db1-291377d7d3b3 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:20.478456 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1aa50ca-4f1b-4970-8db1-291377d7d3b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:20.478758 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1aa50ca-4f1b-4970-8db1-291377d7d3b3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:20.479492 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1aa50ca-4f1b-4970-8db1-291377d7d3b3 None admin] RBAC: Adding query filter params (role.id=0d822a0674b6495fac623877a3b38025, scope.domain.id=default, user.id=8cbbc9233ced43d487079e9479dc555c) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:45:20.479797 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1aa50ca-4f1b-4970-8db1-291377d7d3b3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:20.481253 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1aa50ca-4f1b-4970-8db1-291377d7d3b3 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:20.489401 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 12/20] 10.222.0.46 () {56 vars in 1322 bytes} [Mon Oct 10 11:45:20 2022] GET /identity/v3/role_assignments?user.id=8cbbc9233ced43d487079e9479dc555c&scope.domain.id=default&role.id=0d822a0674b6495fac623877a3b38025 => generated 233 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 11:45:20.508728 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ccf4b4-7ca4-4f8f-beb5-f87d27f3a0e6 None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:20.548231 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99120b79-ba1c-47a4-a146-cc334267b742 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:20.548871 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99120b79-ba1c-47a4-a146-cc334267b742 None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:20.549160 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99120b79-ba1c-47a4-a146-cc334267b742 None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:20.549436 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99120b79-ba1c-47a4-a146-cc334267b742 None admin] PATH_INFO: `/v3/domains` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:20.550120 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99120b79-ba1c-47a4-a146-cc334267b742 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:45:20.550463 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99120b79-ba1c-47a4-a146-cc334267b742 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:20.551956 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99120b79-ba1c-47a4-a146-cc334267b742 None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:20.571691 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 9/21] 10.222.0.46 () {56 vars in 1109 bytes} [Mon Oct 10 11:45:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:45:22.739423 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11156d4f-5f04-4687-ac73-a252898f117f None None] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:22.739938 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11156d4f-5f04-4687-ac73-a252898f117f None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:22.739938 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11156d4f-5f04-4687-ac73-a252898f117f None None] PATH_INFO: `/` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:22.741488 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 13/22] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 10 11:45:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:45:22.747017 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1235cd79-cd04-49e6-922f-6c410f223027 None None] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:22.747446 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1235cd79-cd04-49e6-922f-6c410f223027 None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:22.747636 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1235cd79-cd04-49e6-922f-6c410f223027 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:22.775702 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.auth.core [None req-1235cd79-cd04-49e6-922f-6c410f223027 None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:45:22.776919 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-1235cd79-cd04-49e6-922f-6c410f223027 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:45:22.867945 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 10/23] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 10 11:45:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 10 11:45:23.283200 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63d7ef1-a1cd-44eb-b4ec-0a1b11485f5b None None] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:23.283566 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63d7ef1-a1cd-44eb-b4ec-0a1b11485f5b None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:23.283759 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63d7ef1-a1cd-44eb-b4ec-0a1b11485f5b None None] PATH_INFO: `/` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:23.285539 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 14/24] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 10 11:45:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:45:23.290798 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232aa063-2ee2-419c-824e-a53cd8d28d20 None None] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:23.291094 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232aa063-2ee2-419c-824e-a53cd8d28d20 None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:23.291373 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232aa063-2ee2-419c-824e-a53cd8d28d20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:23.318617 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.auth.core [None req-232aa063-2ee2-419c-824e-a53cd8d28d20 None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:45:23.319894 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-232aa063-2ee2-419c-824e-a53cd8d28d20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:45:23.332403 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd3cc1f-3693-4b14-9248-64ae51d8dbe8 None None] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:23.332995 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd3cc1f-3693-4b14-9248-64ae51d8dbe8 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:23.333482 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd3cc1f-3693-4b14-9248-64ae51d8dbe8 None None] PATH_INFO: `/` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:23.335367 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 15/25] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 10 11:45:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:45:23.342119 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:23.348549 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3da5af-1d03-4b00-a1ff-ed1ccdea6a10 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:23.349076 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3da5af-1d03-4b00-a1ff-ed1ccdea6a10 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:23.349566 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3da5af-1d03-4b00-a1ff-ed1ccdea6a10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:23.373804 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-bf3da5af-1d03-4b00-a1ff-ed1ccdea6a10 None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:45:23.375644 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-bf3da5af-1d03-4b00-a1ff-ed1ccdea6a10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:45:23.431044 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 11/26] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 10 11:45:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 10 11:45:23.435575 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232aa063-2ee2-419c-824e-a53cd8d28d20 None None] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:23.520589 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 16/27] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 10 11:45:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 10 11:45:23.613516 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232aa063-2ee2-419c-824e-a53cd8d28d20 None None] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:23.662779 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559cb108-5a01-4afb-a8f8-7ce93912f994 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:23.663430 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559cb108-5a01-4afb-a8f8-7ce93912f994 None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:23.663715 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559cb108-5a01-4afb-a8f8-7ce93912f994 None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:23.663991 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559cb108-5a01-4afb-a8f8-7ce93912f994 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:23.680934 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559cb108-5a01-4afb-a8f8-7ce93912f994 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 10 11:45:23.685833 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application [None req-559cb108-5a01-4afb-a8f8-7ce93912f994 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 10 11:45:23.685833 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:45:23.685833 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:45:23.685833 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:45:23.685833 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:45:23.685833 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:45:23.685833 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:45:23.685833 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:45:23.685833 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:45:23.685833 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:45:23.685833 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:45:23.685833 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:45:23.685833 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 10 11:45:23.685833 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 10 11:45:23.685833 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 10 11:45:23.685833 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 10 11:45:23.686645 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 10 11:45:23.686645 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 10 11:45:23.686645 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 10 11:45:23.686645 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 10 11:45:23.686645 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:45:23.686645 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:45:23.686645 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:45:23.686645 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:45:23.686645 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:45:23.686645 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:45:23.686645 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:45:23.686645 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application with Lock( Oct 10 11:45:23.686645 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:45:23.686645 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:45:23.686645 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:45:23.686645 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:45:23.687411 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:45:23.687411 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:45:23.687411 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:45:23.687411 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:45:23.687411 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 10 11:45:23.687411 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 10 11:45:23.687411 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 10 11:45:23.687411 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 10 11:45:23.687411 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 10 11:45:23.687411 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 10 11:45:23.687411 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 10 11:45:23.687411 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Oct 10 11:45:23.691828 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 12/28] 10.222.0.46 () {56 vars in 1096 bytes} [Mon Oct 10 11:45:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:45:23.697932 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3da5af-1d03-4b00-a1ff-ed1ccdea6a10 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:23.832508 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559cb108-5a01-4afb-a8f8-7ce93912f994 None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:23.839772 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:23.853230 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3da5af-1d03-4b00-a1ff-ed1ccdea6a10 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:23.870139 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:23.925011 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d756eee0-8700-482c-a750-9b8e505e3bd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:23.926552 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d756eee0-8700-482c-a750-9b8e505e3bd7 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:23.927085 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d756eee0-8700-482c-a750-9b8e505e3bd7 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:23.928087 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d756eee0-8700-482c-a750-9b8e505e3bd7 None admin] PATH_INFO: `/v3/roles` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:23.929261 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d756eee0-8700-482c-a750-9b8e505e3bd7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:45:23.929772 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d756eee0-8700-482c-a750-9b8e505e3bd7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:23.931630 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d756eee0-8700-482c-a750-9b8e505e3bd7 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:23.952010 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 17/29] 10.222.0.46 () {56 vars in 1099 bytes} [Mon Oct 10 11:45:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:45:23.958155 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d756eee0-8700-482c-a750-9b8e505e3bd7 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:24.036886 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559cb108-5a01-4afb-a8f8-7ce93912f994 None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:24.091593 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d756eee0-8700-482c-a750-9b8e505e3bd7 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:24.111174 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad11840-e625-40d7-8fcc-c2b8ba29dd5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:24.112443 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad11840-e625-40d7-8fcc-c2b8ba29dd5c None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:24.112754 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad11840-e625-40d7-8fcc-c2b8ba29dd5c None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:24.113518 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad11840-e625-40d7-8fcc-c2b8ba29dd5c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:24.129126 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad11840-e625-40d7-8fcc-c2b8ba29dd5c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:24.131495 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad11840-e625-40d7-8fcc-c2b8ba29dd5c None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:24.144692 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0895cf2-90af-47d1-9ea2-807bec31ec10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:24.145718 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0895cf2-90af-47d1-9ea2-807bec31ec10 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:24.146077 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0895cf2-90af-47d1-9ea2-807bec31ec10 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:24.146428 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0895cf2-90af-47d1-9ea2-807bec31ec10 None admin] PATH_INFO: `/v3/users/8cbbc9233ced43d487079e9479dc555c` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:24.147728 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0895cf2-90af-47d1-9ea2-807bec31ec10 None admin] RBAC: Authorizing `identity:get_user(user_id=8cbbc9233ced43d487079e9479dc555c)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:24.149262 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0895cf2-90af-47d1-9ea2-807bec31ec10 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:24.151307 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 18/30] 10.222.0.46 () {56 vars in 1177 bytes} [Mon Oct 10 11:45:23 2022] GET /identity/v3/users/8cbbc9233ced43d487079e9479dc555c => generated 247 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 11:45:24.152125 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application [None req-8ad11840-e625-40d7-8fcc-c2b8ba29dd5c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:45:24.152125 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:45:24.152125 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:45:24.152125 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:45:24.152125 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:45:24.152125 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:45:24.152125 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:45:24.152125 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:45:24.152125 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:45:24.152125 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:45:24.152125 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:45:24.152125 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:45:24.152125 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 10 11:45:24.152125 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 10 11:45:24.152125 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 10 11:45:24.152125 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 10 11:45:24.152945 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:45:24.152945 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:45:24.152945 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:45:24.152945 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:45:24.152945 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:45:24.152945 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:45:24.152945 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:45:24.152945 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application with Lock( Oct 10 11:45:24.152945 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:45:24.152945 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:45:24.152945 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:45:24.152945 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:45:24.152945 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:45:24.152945 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:45:24.152945 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:45:24.152945 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:45:24.152945 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 10 11:45:24.153738 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 10 11:45:24.153738 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:45:24.153738 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Oct 10 11:45:24.153898 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0895cf2-90af-47d1-9ea2-807bec31ec10 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:24.155462 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 13/31] 10.222.0.46 () {56 vars in 1108 bytes} [Mon Oct 10 11:45:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 324 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:45:24.156791 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cca884-133b-43ca-b4ad-9dbbeebabd81 None None] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:24.157214 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cca884-133b-43ca-b4ad-9dbbeebabd81 None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:24.157947 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cca884-133b-43ca-b4ad-9dbbeebabd81 None None] PATH_INFO: `/` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:24.161205 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 14/32] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 10 11:45:24 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:45:24.166002 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cca884-133b-43ca-b4ad-9dbbeebabd81 None None] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:24.189308 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:24.248243 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:24.287253 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cca884-133b-43ca-b4ad-9dbbeebabd81 None None] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:24.288341 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0895cf2-90af-47d1-9ea2-807bec31ec10 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:24.324073 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e959a6b-11d0-444b-bdaf-4a3826216b58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:24.325051 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e959a6b-11d0-444b-bdaf-4a3826216b58 None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:24.325474 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e959a6b-11d0-444b-bdaf-4a3826216b58 None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:24.325924 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e959a6b-11d0-444b-bdaf-4a3826216b58 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:24.327311 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e959a6b-11d0-444b-bdaf-4a3826216b58 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:24.328975 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e959a6b-11d0-444b-bdaf-4a3826216b58 None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:24.330735 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 15/33] 10.222.0.46 () {56 vars in 1108 bytes} [Mon Oct 10 11:45:24 2022] GET /identity/v3/domains/default => generated 199 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:45:24.332333 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e959a6b-11d0-444b-bdaf-4a3826216b58 None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:24.343101 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061d580d-7480-4714-9d84-cc9e8e9ecb3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:24.343737 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061d580d-7480-4714-9d84-cc9e8e9ecb3f None admin] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:24.344022 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061d580d-7480-4714-9d84-cc9e8e9ecb3f None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:24.344303 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061d580d-7480-4714-9d84-cc9e8e9ecb3f None admin] PATH_INFO: `/v3/roles` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:24.344998 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061d580d-7480-4714-9d84-cc9e8e9ecb3f None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:24.346410 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061d580d-7480-4714-9d84-cc9e8e9ecb3f None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:24.444900 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e959a6b-11d0-444b-bdaf-4a3826216b58 None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:24.486196 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62075e6-04db-46e6-8e8b-36b36ef264ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:24.487117 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62075e6-04db-46e6-8e8b-36b36ef264ea None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:24.487593 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62075e6-04db-46e6-8e8b-36b36ef264ea None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:24.488125 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62075e6-04db-46e6-8e8b-36b36ef264ea None admin] PATH_INFO: `/v3/domains` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:24.488967 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62075e6-04db-46e6-8e8b-36b36ef264ea None admin] RBAC: Adding query filter params (name=Default) {{(pid=81676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:45:24.489460 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62075e6-04db-46e6-8e8b-36b36ef264ea None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:24.491077 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62075e6-04db-46e6-8e8b-36b36ef264ea None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:24.498160 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:24.504959 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 16/34] 10.222.0.46 () {56 vars in 1109 bytes} [Mon Oct 10 11:45:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:45:24.506500 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376e6705-0369-498b-9257-81a4c9fd99bc None None] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:24.506970 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376e6705-0369-498b-9257-81a4c9fd99bc None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:24.507412 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376e6705-0369-498b-9257-81a4c9fd99bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:24.527551 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.auth.core [None req-376e6705-0369-498b-9257-81a4c9fd99bc None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:45:24.528991 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-376e6705-0369-498b-9257-81a4c9fd99bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:45:24.606428 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 17/35] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 10 11:45:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 10 11:45:24.607839 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376e6705-0369-498b-9257-81a4c9fd99bc None None] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:24.633958 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 19/36] 10.222.0.46 () {60 vars in 1131 bytes} [Mon Oct 10 11:45:24 2022] POST /identity/v3/roles => generated 219 bytes in 482 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 11:45:24.640096 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061d580d-7480-4714-9d84-cc9e8e9ecb3f None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:24.740269 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376e6705-0369-498b-9257-81a4c9fd99bc None None] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:24.775071 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:24.790652 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163e1065-60e8-4df4-9473-79b1eecb0104 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:24.791672 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163e1065-60e8-4df4-9473-79b1eecb0104 None admin] REQUEST_METHOD: `PUT` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:24.792045 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163e1065-60e8-4df4-9473-79b1eecb0104 None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:24.792396 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163e1065-60e8-4df4-9473-79b1eecb0104 None admin] PATH_INFO: `/v3/domains/default/users/8cbbc9233ced43d487079e9479dc555c/roles/0d822a0674b6495fac623877a3b38025` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:24.794791 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163e1065-60e8-4df4-9473-79b1eecb0104 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=8cbbc9233ced43d487079e9479dc555c, role_id=0d822a0674b6495fac623877a3b38025)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:24.797064 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163e1065-60e8-4df4-9473-79b1eecb0104 None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:24.836184 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:24.854701 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061d580d-7480-4714-9d84-cc9e8e9ecb3f None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:24.893589 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1566d766-c496-41d3-8130-a3851e9303e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:24.894844 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1566d766-c496-41d3-8130-a3851e9303e8 None admin] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:24.895129 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1566d766-c496-41d3-8130-a3851e9303e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:24.895403 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1566d766-c496-41d3-8130-a3851e9303e8 None admin] PATH_INFO: `/v3/projects` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:24.896069 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1566d766-c496-41d3-8130-a3851e9303e8 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:24.897741 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1566d766-c496-41d3-8130-a3851e9303e8 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:25.005443 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 20/37] 10.222.0.46 () {60 vars in 1141 bytes} [Mon Oct 10 11:45:24 2022] POST /identity/v3/projects => generated 301 bytes in 365 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 11:45:25.006345 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0375e4-fe39-4bbe-8319-8fe5d6b16611 None None] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:25.007118 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0375e4-fe39-4bbe-8319-8fe5d6b16611 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:25.007830 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0375e4-fe39-4bbe-8319-8fe5d6b16611 None None] PATH_INFO: `/` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:25.010226 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 21/38] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 10 11:45:25 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:45:25.013950 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49343cad-1b7d-43a9-b2c1-34d296e3dedf None None] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:25.013950 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49343cad-1b7d-43a9-b2c1-34d296e3dedf None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:25.013950 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49343cad-1b7d-43a9-b2c1-34d296e3dedf None None] PATH_INFO: `/` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:25.013950 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 22/39] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 10 11:45:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:45:25.014527 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49343cad-1b7d-43a9-b2c1-34d296e3dedf None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:25.119426 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 18/40] 10.222.0.46 () {58 vars in 1361 bytes} [Mon Oct 10 11:45:24 2022] PUT /identity/v3/domains/default/users/8cbbc9233ced43d487079e9479dc555c/roles/0d822a0674b6495fac623877a3b38025 => generated 0 bytes in 512 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 11:45:25.121180 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c668029-755f-41db-a90c-2a120730e757 None None] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:25.121476 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c668029-755f-41db-a90c-2a120730e757 None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:25.121755 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c668029-755f-41db-a90c-2a120730e757 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:25.142330 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.auth.core [None req-0c668029-755f-41db-a90c-2a120730e757 None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:45:25.144073 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-0c668029-755f-41db-a90c-2a120730e757 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:45:25.147884 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49343cad-1b7d-43a9-b2c1-34d296e3dedf None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:25.196901 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb84fea7-eaea-4898-9fac-47d693d1576f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:25.198281 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb84fea7-eaea-4898-9fac-47d693d1576f None admin] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:25.198567 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb84fea7-eaea-4898-9fac-47d693d1576f None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:25.198865 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb84fea7-eaea-4898-9fac-47d693d1576f None admin] PATH_INFO: `/v3/roles` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:25.199569 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb84fea7-eaea-4898-9fac-47d693d1576f None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:25.202500 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb84fea7-eaea-4898-9fac-47d693d1576f None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:25.236737 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 19/41] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 10 11:45:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 10 11:45:25.238604 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6bb61f-c067-463e-90e7-02d575da19ad None None] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:25.238930 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6bb61f-c067-463e-90e7-02d575da19ad None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:25.239230 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6bb61f-c067-463e-90e7-02d575da19ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:25.259445 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.auth.core [None req-2c6bb61f-c067-463e-90e7-02d575da19ad None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:45:25.260624 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-2c6bb61f-c067-463e-90e7-02d575da19ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:45:25.277111 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:25.283300 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 23/42] 10.222.0.46 () {60 vars in 1131 bytes} [Mon Oct 10 11:45:25 2022] POST /identity/v3/roles => generated 225 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 11:45:25.355767 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 20/43] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 10 11:45:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 10 11:45:25.523348 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb84fea7-eaea-4898-9fac-47d693d1576f None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:25.635763 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6bb61f-c067-463e-90e7-02d575da19ad None None] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:25.635966 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb84fea7-eaea-4898-9fac-47d693d1576f None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:25.674446 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef510aa8-e2f9-4490-9999-a3c8e45c5fcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:25.674446 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef510aa8-e2f9-4490-9999-a3c8e45c5fcb None admin] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:25.674813 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef510aa8-e2f9-4490-9999-a3c8e45c5fcb None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:25.675089 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef510aa8-e2f9-4490-9999-a3c8e45c5fcb None admin] PATH_INFO: `/v3/roles` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:25.675770 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef510aa8-e2f9-4490-9999-a3c8e45c5fcb None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:25.677191 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef510aa8-e2f9-4490-9999-a3c8e45c5fcb None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:25.687024 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:25.741602 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6bb61f-c067-463e-90e7-02d575da19ad None None] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:25.745074 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 24/44] 10.222.0.46 () {60 vars in 1131 bytes} [Mon Oct 10 11:45:25 2022] POST /identity/v3/roles => generated 223 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 11:45:25.781680 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8096421d-f2ab-4b66-ac89-90aceede05a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:25.782275 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8096421d-f2ab-4b66-ac89-90aceede05a2 None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:25.782563 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8096421d-f2ab-4b66-ac89-90aceede05a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:25.783060 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8096421d-f2ab-4b66-ac89-90aceede05a2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:25.784287 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8096421d-f2ab-4b66-ac89-90aceede05a2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:25.786598 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8096421d-f2ab-4b66-ac89-90aceede05a2 None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:25.794752 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 21/45] 10.222.0.46 () {56 vars in 1108 bytes} [Mon Oct 10 11:45:25 2022] GET /identity/v3/domains/default => generated 199 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:45:25.799146 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef510aa8-e2f9-4490-9999-a3c8e45c5fcb None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:25.866705 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:25.914740 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef510aa8-e2f9-4490-9999-a3c8e45c5fcb None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:25.953338 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4da0d2-2ad0-4554-825c-4b6cdefcef48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:25.953972 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4da0d2-2ad0-4554-825c-4b6cdefcef48 None admin] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:25.954261 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4da0d2-2ad0-4554-825c-4b6cdefcef48 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:25.954542 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4da0d2-2ad0-4554-825c-4b6cdefcef48 None admin] PATH_INFO: `/v3/projects` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:25.955235 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4da0d2-2ad0-4554-825c-4b6cdefcef48 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:25.957244 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4da0d2-2ad0-4554-825c-4b6cdefcef48 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:26.042558 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 25/46] 10.222.0.46 () {60 vars in 1141 bytes} [Mon Oct 10 11:45:25 2022] POST /identity/v3/projects => generated 312 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 11:45:27.375366 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938547b5-d63d-4a50-90ec-f44bfff77604 None None] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:27.375881 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938547b5-d63d-4a50-90ec-f44bfff77604 None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:27.375963 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938547b5-d63d-4a50-90ec-f44bfff77604 None None] PATH_INFO: `/` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:27.377295 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 22/47] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 10 11:45:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:45:27.382738 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1992e5e5-5547-40a8-bbb8-3dcae01dc63b None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:27.383146 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1992e5e5-5547-40a8-bbb8-3dcae01dc63b None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:27.383307 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1992e5e5-5547-40a8-bbb8-3dcae01dc63b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:27.401310 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-1992e5e5-5547-40a8-bbb8-3dcae01dc63b None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:45:27.402860 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-1992e5e5-5547-40a8-bbb8-3dcae01dc63b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:45:27.471621 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 26/48] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 10 11:45:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 10 11:45:27.741756 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938547b5-d63d-4a50-90ec-f44bfff77604 None None] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:27.843524 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938547b5-d63d-4a50-90ec-f44bfff77604 None None] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:27.856531 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:27.877767 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a722d2cf-d5e2-4679-ae51-c41579b5932e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:27.879062 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a722d2cf-d5e2-4679-ae51-c41579b5932e None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:27.879347 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a722d2cf-d5e2-4679-ae51-c41579b5932e None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:27.879622 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a722d2cf-d5e2-4679-ae51-c41579b5932e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:27.889958 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a722d2cf-d5e2-4679-ae51-c41579b5932e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 10 11:45:27.894843 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application [None req-a722d2cf-d5e2-4679-ae51-c41579b5932e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 10 11:45:27.894843 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:45:27.894843 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:45:27.894843 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:45:27.894843 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:45:27.894843 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:45:27.894843 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:45:27.894843 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:45:27.894843 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:45:27.894843 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:45:27.894843 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:45:27.894843 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:45:27.894843 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 10 11:45:27.894843 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 10 11:45:27.894843 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 10 11:45:27.894843 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 10 11:45:27.895654 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 10 11:45:27.895654 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 10 11:45:27.895654 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 10 11:45:27.895654 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 10 11:45:27.895654 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:45:27.895654 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:45:27.895654 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:45:27.895654 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:45:27.895654 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:45:27.895654 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:45:27.895654 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:45:27.895654 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application with Lock( Oct 10 11:45:27.895654 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:45:27.895654 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:45:27.895654 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:45:27.895654 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:45:27.896403 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:45:27.896403 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:45:27.896403 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:45:27.896403 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:45:27.896403 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 10 11:45:27.896403 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 10 11:45:27.896403 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 10 11:45:27.896403 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 10 11:45:27.896403 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 10 11:45:27.896403 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 10 11:45:27.896403 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 10 11:45:27.896403 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Oct 10 11:45:27.896403 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 23/49] 10.222.0.46 () {56 vars in 1096 bytes} [Mon Oct 10 11:45:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:45:27.901865 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1992e5e5-5547-40a8-bbb8-3dcae01dc63b None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:27.944669 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:28.001232 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1992e5e5-5547-40a8-bbb8-3dcae01dc63b None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:28.035585 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ce031b-5bd9-4e5a-a97b-4da6daac1b4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:28.036205 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ce031b-5bd9-4e5a-a97b-4da6daac1b4a None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:28.036495 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ce031b-5bd9-4e5a-a97b-4da6daac1b4a None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:28.036778 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ce031b-5bd9-4e5a-a97b-4da6daac1b4a None admin] PATH_INFO: `/v3/roles` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:28.037456 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ce031b-5bd9-4e5a-a97b-4da6daac1b4a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:45:28.037768 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ce031b-5bd9-4e5a-a97b-4da6daac1b4a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:28.039194 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ce031b-5bd9-4e5a-a97b-4da6daac1b4a None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:28.047318 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 27/50] 10.222.0.46 () {56 vars in 1099 bytes} [Mon Oct 10 11:45:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:45:28.053562 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a722d2cf-d5e2-4679-ae51-c41579b5932e None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:28.158238 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a722d2cf-d5e2-4679-ae51-c41579b5932e None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:28.193193 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd596b1-756e-4f6f-bf31-2981764091d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:28.193916 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd596b1-756e-4f6f-bf31-2981764091d9 None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:28.194266 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd596b1-756e-4f6f-bf31-2981764091d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:28.194550 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd596b1-756e-4f6f-bf31-2981764091d9 None admin] PATH_INFO: `/v3/users/8cbbc9233ced43d487079e9479dc555c` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:28.195705 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd596b1-756e-4f6f-bf31-2981764091d9 None admin] RBAC: Authorizing `identity:get_user(user_id=8cbbc9233ced43d487079e9479dc555c)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:28.197158 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd596b1-756e-4f6f-bf31-2981764091d9 None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:28.198989 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 24/51] 10.222.0.46 () {56 vars in 1177 bytes} [Mon Oct 10 11:45:28 2022] GET /identity/v3/users/8cbbc9233ced43d487079e9479dc555c => generated 247 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:45:28.204190 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ce031b-5bd9-4e5a-a97b-4da6daac1b4a None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:28.206872 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93845f2-9109-47d7-b848-67e99c301d4f None None] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:28.207163 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93845f2-9109-47d7-b848-67e99c301d4f None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:28.207445 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93845f2-9109-47d7-b848-67e99c301d4f None None] PATH_INFO: `/` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:28.208733 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 25/52] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 10 11:45:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:45:28.213812 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616400a9-6ae0-4bfc-b74d-9b4a6f5e1734 None None] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:28.214110 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616400a9-6ae0-4bfc-b74d-9b4a6f5e1734 None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:28.214389 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616400a9-6ae0-4bfc-b74d-9b4a6f5e1734 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:28.232625 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.auth.core [None req-616400a9-6ae0-4bfc-b74d-9b4a6f5e1734 None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:45:28.233869 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-616400a9-6ae0-4bfc-b74d-9b4a6f5e1734 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:45:28.269036 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:28.302096 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 26/53] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 10 11:45:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 10 11:45:28.305522 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ce031b-5bd9-4e5a-a97b-4da6daac1b4a None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:28.307087 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:28.339247 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be4c96d-bcac-45f6-8b15-130a4b1894fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:28.340325 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be4c96d-bcac-45f6-8b15-130a4b1894fe None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:28.340651 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be4c96d-bcac-45f6-8b15-130a4b1894fe None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:28.340990 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be4c96d-bcac-45f6-8b15-130a4b1894fe None admin] PATH_INFO: `/v3/domains/default` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:28.342124 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be4c96d-bcac-45f6-8b15-130a4b1894fe None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:28.343688 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be4c96d-bcac-45f6-8b15-130a4b1894fe None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:28.345260 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 28/54] 10.222.0.46 () {56 vars in 1108 bytes} [Mon Oct 10 11:45:28 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:45:28.351627 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616400a9-6ae0-4bfc-b74d-9b4a6f5e1734 None None] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:28.453554 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616400a9-6ae0-4bfc-b74d-9b4a6f5e1734 None None] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:28.489744 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad49274-3461-45be-8c98-dfaaf33d1557 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:28.490361 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad49274-3461-45be-8c98-dfaaf33d1557 None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:28.490641 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad49274-3461-45be-8c98-dfaaf33d1557 None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:28.490949 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad49274-3461-45be-8c98-dfaaf33d1557 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:28.491678 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad49274-3461-45be-8c98-dfaaf33d1557 None admin] RBAC: Adding query filter params (role.id=0d822a0674b6495fac623877a3b38025, scope.domain.id=default, user.id=8cbbc9233ced43d487079e9479dc555c) {{(pid=81676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:45:28.491983 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad49274-3461-45be-8c98-dfaaf33d1557 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:28.493424 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad49274-3461-45be-8c98-dfaaf33d1557 None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:28.500684 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 27/55] 10.222.0.46 () {56 vars in 1322 bytes} [Mon Oct 10 11:45:28 2022] GET /identity/v3/role_assignments?user.id=8cbbc9233ced43d487079e9479dc555c&scope.domain.id=default&role.id=0d822a0674b6495fac623877a3b38025 => generated 530 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 11:45:28.580804 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be4c96d-bcac-45f6-8b15-130a4b1894fe None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:28.691513 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be4c96d-bcac-45f6-8b15-130a4b1894fe None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:28.726734 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd14a25-fb33-44fc-af04-06a14f5c5399 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:28.727361 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd14a25-fb33-44fc-af04-06a14f5c5399 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:28.727647 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd14a25-fb33-44fc-af04-06a14f5c5399 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:28.727928 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd14a25-fb33-44fc-af04-06a14f5c5399 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:28.729071 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd14a25-fb33-44fc-af04-06a14f5c5399 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:28.730529 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd14a25-fb33-44fc-af04-06a14f5c5399 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:28.732275 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 29/56] 10.222.0.46 () {56 vars in 1108 bytes} [Mon Oct 10 11:45:28 2022] GET /identity/v3/domains/default => generated 199 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:45:28.738493 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad49274-3461-45be-8c98-dfaaf33d1557 None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:28.774588 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:28.838712 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad49274-3461-45be-8c98-dfaaf33d1557 None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:28.872239 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ef908a-db68-4179-8a1c-92dfdccbd78f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:28.872845 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ef908a-db68-4179-8a1c-92dfdccbd78f None admin] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:28.873130 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ef908a-db68-4179-8a1c-92dfdccbd78f None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:28.873408 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ef908a-db68-4179-8a1c-92dfdccbd78f None admin] PATH_INFO: `/v3/projects` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:28.874067 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ef908a-db68-4179-8a1c-92dfdccbd78f None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:28.875826 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ef908a-db68-4179-8a1c-92dfdccbd78f None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:28.963166 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 28/57] 10.222.0.46 () {60 vars in 1140 bytes} [Mon Oct 10 11:45:28 2022] POST /identity/v3/projects => generated 298 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 11:45:31.104140 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2d1834-d116-42e2-9b1e-5d6eb9d8bec6 None None] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:31.104649 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2d1834-d116-42e2-9b1e-5d6eb9d8bec6 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:31.104761 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2d1834-d116-42e2-9b1e-5d6eb9d8bec6 None None] PATH_INFO: `/` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:31.106071 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 30/58] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 10 11:45:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:45:31.111390 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c441049-7661-4484-a907-b24398e4d4b2 None None] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:31.111783 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c441049-7661-4484-a907-b24398e4d4b2 None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:31.111948 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c441049-7661-4484-a907-b24398e4d4b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:31.130209 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.auth.core [None req-8c441049-7661-4484-a907-b24398e4d4b2 None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:45:31.131425 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-8c441049-7661-4484-a907-b24398e4d4b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:45:31.195143 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 29/59] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 10 11:45:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 10 11:45:31.460785 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2d1834-d116-42e2-9b1e-5d6eb9d8bec6 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:31.601579 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:31.661953 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2d1834-d116-42e2-9b1e-5d6eb9d8bec6 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:31.694728 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8554bfd3-16d9-49a1-a595-bee16816990c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:31.695358 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8554bfd3-16d9-49a1-a595-bee16816990c None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:31.695645 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8554bfd3-16d9-49a1-a595-bee16816990c None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:31.695928 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8554bfd3-16d9-49a1-a595-bee16816990c None admin] PATH_INFO: `/v3/domains/default` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:31.697050 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8554bfd3-16d9-49a1-a595-bee16816990c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:31.698481 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8554bfd3-16d9-49a1-a595-bee16816990c None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:31.700089 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 31/60] 10.222.0.46 () {56 vars in 1108 bytes} [Mon Oct 10 11:45:31 2022] GET /identity/v3/domains/default => generated 199 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:45:31.705836 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c441049-7661-4484-a907-b24398e4d4b2 None None] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:31.722949 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:31.802136 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c441049-7661-4484-a907-b24398e4d4b2 None None] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:31.835734 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8be7e57-9449-4022-a4cf-ae58099c2cd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:31.836333 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8be7e57-9449-4022-a4cf-ae58099c2cd1 None admin] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:31.836618 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8be7e57-9449-4022-a4cf-ae58099c2cd1 None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:31.836891 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8be7e57-9449-4022-a4cf-ae58099c2cd1 None admin] PATH_INFO: `/v3/users` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:31.837610 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8be7e57-9449-4022-a4cf-ae58099c2cd1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:31.839042 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8be7e57-9449-4022-a4cf-ae58099c2cd1 None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:31.960879 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 30/61] 10.222.0.46 () {60 vars in 1132 bytes} [Mon Oct 10 11:45:31 2022] POST /identity/v3/users => generated 275 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 11:45:35.433307 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8533e91c-b879-4480-8fc9-d3ec9d70838a None None] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:35.434186 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8533e91c-b879-4480-8fc9-d3ec9d70838a None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:35.434609 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8533e91c-b879-4480-8fc9-d3ec9d70838a None None] PATH_INFO: `/` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:35.436594 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 32/62] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 10 11:45:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:45:35.442980 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ccebe4-2b3a-431b-bdeb-89806500a3e7 None None] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:35.443847 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ccebe4-2b3a-431b-bdeb-89806500a3e7 None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:35.444378 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ccebe4-2b3a-431b-bdeb-89806500a3e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:35.470585 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1742ff-ca12-48f8-b48c-64deceb9fd6e None None] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:35.471313 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1742ff-ca12-48f8-b48c-64deceb9fd6e None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:35.472399 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1742ff-ca12-48f8-b48c-64deceb9fd6e None None] PATH_INFO: `/` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:35.474202 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 33/63] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 10 11:45:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:45:35.474324 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.auth.core [None req-b8ccebe4-2b3a-431b-bdeb-89806500a3e7 None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:45:35.475506 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-b8ccebe4-2b3a-431b-bdeb-89806500a3e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:45:35.480267 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a50f82-a0bb-450b-a80c-477db10e78f3 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:35.480818 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a50f82-a0bb-450b-a80c-477db10e78f3 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:35.481288 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a50f82-a0bb-450b-a80c-477db10e78f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:35.523806 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-42a50f82-a0bb-450b-a80c-477db10e78f3 None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:45:35.525341 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-42a50f82-a0bb-450b-a80c-477db10e78f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:45:35.569842 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 31/64] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 10 11:45:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 10 11:45:35.575408 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb9427c-7b01-4021-b542-807df95b50b1 None None] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:35.575702 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb9427c-7b01-4021-b542-807df95b50b1 None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:35.575985 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb9427c-7b01-4021-b542-807df95b50b1 None None] PATH_INFO: `/` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:35.581091 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 32/65] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 10 11:45:35 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:45:35.589821 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0b3fd2-850c-41ae-947e-b559e6a6dbab None None] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:35.590101 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0b3fd2-850c-41ae-947e-b559e6a6dbab None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:35.590445 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0b3fd2-850c-41ae-947e-b559e6a6dbab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:35.607070 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:35.623493 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.auth.core [None req-cf0b3fd2-850c-41ae-947e-b559e6a6dbab None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:45:35.628613 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-cf0b3fd2-850c-41ae-947e-b559e6a6dbab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:45:35.658360 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 34/66] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 10 11:45:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 10 11:45:35.660241 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bafde59-fd71-43a3-b83d-84134bf339bf None None] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:35.660530 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bafde59-fd71-43a3-b83d-84134bf339bf None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:35.660806 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bafde59-fd71-43a3-b83d-84134bf339bf None None] PATH_INFO: `/` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:35.662408 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 35/67] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 10 11:45:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:45:35.670374 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d746ca-c53e-4d5e-968d-5fd8081509ec None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:35.670681 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d746ca-c53e-4d5e-968d-5fd8081509ec None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:35.670964 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d746ca-c53e-4d5e-968d-5fd8081509ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:35.697240 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-54d746ca-c53e-4d5e-968d-5fd8081509ec None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:45:35.698348 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-54d746ca-c53e-4d5e-968d-5fd8081509ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:45:35.763063 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:35.766943 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 33/68] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 10 11:45:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 10 11:45:35.834528 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 36/69] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 10 11:45:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 10 11:45:35.893509 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0b3fd2-850c-41ae-947e-b559e6a6dbab None None] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:35.913771 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c5c869-6928-41fe-b0b7-654a41d1c0b6 None None] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:35.914468 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c5c869-6928-41fe-b0b7-654a41d1c0b6 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:35.914994 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c5c869-6928-41fe-b0b7-654a41d1c0b6 None None] PATH_INFO: `/` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:35.916939 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 37/70] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 10 11:45:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 10 11:45:35.922776 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3637a2e-01b9-4d0e-9a6e-c38ed0368fd9 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:35.923327 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3637a2e-01b9-4d0e-9a6e-c38ed0368fd9 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:35.923806 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3637a2e-01b9-4d0e-9a6e-c38ed0368fd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:35.947378 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-b3637a2e-01b9-4d0e-9a6e-c38ed0368fd9 None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:45:35.948890 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-b3637a2e-01b9-4d0e-9a6e-c38ed0368fd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:45:36.025899 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 38/71] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 10 11:45:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 10 11:45:36.027101 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3637a2e-01b9-4d0e-9a6e-c38ed0368fd9 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:36.032991 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0b3fd2-850c-41ae-947e-b559e6a6dbab None None] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:36.085868 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40becd42-1bbd-48e2-b1fa-f6256797204e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:36.086923 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40becd42-1bbd-48e2-b1fa-f6256797204e None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:36.087419 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40becd42-1bbd-48e2-b1fa-f6256797204e None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:36.087884 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40becd42-1bbd-48e2-b1fa-f6256797204e None admin] PATH_INFO: `/v3/roles/member` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:36.088875 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:36.100081 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40becd42-1bbd-48e2-b1fa-f6256797204e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 10 11:45:36.107663 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application [None req-40becd42-1bbd-48e2-b1fa-f6256797204e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 10 11:45:36.107663 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:45:36.107663 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:45:36.107663 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:45:36.107663 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:45:36.107663 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:45:36.107663 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:45:36.107663 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:45:36.107663 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:45:36.107663 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:45:36.107663 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:45:36.107663 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:45:36.107663 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 10 11:45:36.107663 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 10 11:45:36.107663 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 10 11:45:36.107663 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 10 11:45:36.108513 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 10 11:45:36.108513 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 10 11:45:36.108513 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 10 11:45:36.108513 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 10 11:45:36.108513 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:45:36.108513 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:45:36.108513 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:45:36.108513 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:45:36.108513 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:45:36.108513 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:45:36.108513 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:45:36.108513 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application with Lock( Oct 10 11:45:36.108513 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:45:36.108513 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:45:36.108513 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:45:36.108513 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:45:36.109246 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:45:36.109246 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:45:36.109246 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:45:36.109246 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:45:36.109246 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 10 11:45:36.109246 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 10 11:45:36.109246 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 10 11:45:36.109246 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 10 11:45:36.109246 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 10 11:45:36.109246 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 10 11:45:36.109246 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 10 11:45:36.109246 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Oct 10 11:45:36.112723 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 34/72] 10.222.0.46 () {56 vars in 1099 bytes} [Mon Oct 10 11:45:35 2022] GET /identity/v3/roles/member => generated 84 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:45:36.114628 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40becd42-1bbd-48e2-b1fa-f6256797204e None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:36.177148 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:36.183702 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3637a2e-01b9-4d0e-9a6e-c38ed0368fd9 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:36.222746 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92d43bd-9e28-4e70-996e-479ce4fc92e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:36.223436 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92d43bd-9e28-4e70-996e-479ce4fc92e0 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:36.223996 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92d43bd-9e28-4e70-996e-479ce4fc92e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:36.224495 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92d43bd-9e28-4e70-996e-479ce4fc92e0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:36.231249 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40becd42-1bbd-48e2-b1fa-f6256797204e None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:36.236769 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92d43bd-9e28-4e70-996e-479ce4fc92e0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 10 11:45:36.246871 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application [None req-a92d43bd-9e28-4e70-996e-479ce4fc92e0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 10 11:45:36.246871 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:45:36.246871 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:45:36.246871 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:45:36.246871 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:45:36.246871 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:45:36.246871 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:45:36.246871 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:45:36.246871 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:45:36.246871 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:45:36.246871 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:45:36.246871 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:45:36.246871 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 10 11:45:36.246871 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 10 11:45:36.246871 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 10 11:45:36.246871 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 10 11:45:36.247726 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 10 11:45:36.247726 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 10 11:45:36.247726 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 10 11:45:36.247726 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 10 11:45:36.247726 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:45:36.247726 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:45:36.247726 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:45:36.247726 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:45:36.247726 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:45:36.247726 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:45:36.247726 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:45:36.247726 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application with Lock( Oct 10 11:45:36.247726 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:45:36.247726 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:45:36.247726 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:45:36.247726 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:45:36.248487 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:45:36.248487 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:45:36.248487 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:45:36.248487 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:45:36.248487 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 10 11:45:36.248487 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 10 11:45:36.248487 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 10 11:45:36.248487 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 10 11:45:36.248487 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 10 11:45:36.248487 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 10 11:45:36.248487 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 10 11:45:36.248487 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Oct 10 11:45:36.250696 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 39/73] 10.222.0.46 () {56 vars in 1096 bytes} [Mon Oct 10 11:45:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:45:36.251663 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92d43bd-9e28-4e70-996e-479ce4fc92e0 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:36.271312 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5687998-102b-46ca-82a6-0e613c73141f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:36.271952 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5687998-102b-46ca-82a6-0e613c73141f None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:36.272237 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5687998-102b-46ca-82a6-0e613c73141f None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:36.272514 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5687998-102b-46ca-82a6-0e613c73141f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:36.283070 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5687998-102b-46ca-82a6-0e613c73141f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 10 11:45:36.288696 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application [None req-a5687998-102b-46ca-82a6-0e613c73141f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 10 11:45:36.288696 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:45:36.288696 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:45:36.288696 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:45:36.288696 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:45:36.288696 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:45:36.288696 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:45:36.288696 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:45:36.288696 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:45:36.288696 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:45:36.288696 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:45:36.288696 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:45:36.288696 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 10 11:45:36.288696 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 10 11:45:36.288696 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 10 11:45:36.288696 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 10 11:45:36.289545 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 10 11:45:36.289545 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 10 11:45:36.289545 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 10 11:45:36.289545 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 10 11:45:36.289545 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:45:36.289545 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:45:36.289545 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:45:36.289545 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:45:36.289545 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:45:36.289545 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:45:36.289545 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:45:36.289545 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application with Lock( Oct 10 11:45:36.289545 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:45:36.289545 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:45:36.289545 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:45:36.289545 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:45:36.290295 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:45:36.290295 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:45:36.290295 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:45:36.290295 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:45:36.290295 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 10 11:45:36.290295 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 10 11:45:36.290295 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 10 11:45:36.290295 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 10 11:45:36.290295 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 10 11:45:36.290295 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 10 11:45:36.290295 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 10 11:45:36.290295 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Oct 10 11:45:36.290295 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 35/74] 10.222.0.46 () {56 vars in 1114 bytes} [Mon Oct 10 11:45:36 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:45:36.290984 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5687998-102b-46ca-82a6-0e613c73141f None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:36.353188 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:36.364453 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92d43bd-9e28-4e70-996e-479ce4fc92e0 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:36.413775 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811e52cd-57c5-40e7-b337-764869889507 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:36.414707 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811e52cd-57c5-40e7-b337-764869889507 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:36.415179 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811e52cd-57c5-40e7-b337-764869889507 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:36.415567 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811e52cd-57c5-40e7-b337-764869889507 None admin] PATH_INFO: `/v3/roles` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:36.416301 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811e52cd-57c5-40e7-b337-764869889507 None admin] RBAC: Adding query filter params (name=member) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:45:36.416703 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811e52cd-57c5-40e7-b337-764869889507 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:36.417632 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5687998-102b-46ca-82a6-0e613c73141f None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:36.418470 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811e52cd-57c5-40e7-b337-764869889507 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:36.432134 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 40/75] 10.222.0.46 () {56 vars in 1101 bytes} [Mon Oct 10 11:45:36 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:45:36.433534 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811e52cd-57c5-40e7-b337-764869889507 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:36.455267 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:36.459100 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5644ab3c-cf1f-404a-bdfd-ecbb835673fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:36.465301 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5644ab3c-cf1f-404a-bdfd-ecbb835673fd None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:36.465301 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5644ab3c-cf1f-404a-bdfd-ecbb835673fd None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:36.465301 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5644ab3c-cf1f-404a-bdfd-ecbb835673fd None admin] PATH_INFO: `/v3/roles/member` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:36.471909 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5644ab3c-cf1f-404a-bdfd-ecbb835673fd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 10 11:45:36.477227 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application [None req-5644ab3c-cf1f-404a-bdfd-ecbb835673fd None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 10 11:45:36.477227 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:45:36.477227 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:45:36.477227 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:45:36.477227 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:45:36.477227 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:45:36.477227 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:45:36.477227 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:45:36.477227 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:45:36.477227 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:45:36.477227 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:45:36.477227 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:45:36.477227 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 10 11:45:36.477227 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 10 11:45:36.477227 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 10 11:45:36.477227 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 10 11:45:36.478050 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 10 11:45:36.478050 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 10 11:45:36.478050 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 10 11:45:36.478050 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 10 11:45:36.478050 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:45:36.478050 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:45:36.478050 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:45:36.478050 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:45:36.478050 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:45:36.478050 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:45:36.478050 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:45:36.478050 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application with Lock( Oct 10 11:45:36.478050 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:45:36.478050 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:45:36.478050 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:45:36.478050 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:45:36.479007 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:45:36.479007 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:45:36.479007 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:45:36.479007 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:45:36.479007 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 10 11:45:36.479007 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 10 11:45:36.479007 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 10 11:45:36.479007 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 10 11:45:36.479007 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 10 11:45:36.479007 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 10 11:45:36.479007 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 10 11:45:36.479007 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Oct 10 11:45:36.479007 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 36/76] 10.222.0.46 () {56 vars in 1099 bytes} [Mon Oct 10 11:45:36 2022] GET /identity/v3/roles/member => generated 84 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:45:36.480713 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5644ab3c-cf1f-404a-bdfd-ecbb835673fd None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:36.547034 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811e52cd-57c5-40e7-b337-764869889507 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:36.580076 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5644ab3c-cf1f-404a-bdfd-ecbb835673fd None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:36.580388 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1120e880-8ae1-4aa7-9b41-86521e23ffa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:36.581554 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1120e880-8ae1-4aa7-9b41-86521e23ffa5 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:36.581835 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1120e880-8ae1-4aa7-9b41-86521e23ffa5 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:36.582116 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1120e880-8ae1-4aa7-9b41-86521e23ffa5 None admin] PATH_INFO: `/v3/roles` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:36.582819 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1120e880-8ae1-4aa7-9b41-86521e23ffa5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:45:36.583123 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1120e880-8ae1-4aa7-9b41-86521e23ffa5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:36.584500 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1120e880-8ae1-4aa7-9b41-86521e23ffa5 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:36.592689 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 41/77] 10.222.0.46 () {56 vars in 1099 bytes} [Mon Oct 10 11:45:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:45:36.593434 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:36.593538 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1120e880-8ae1-4aa7-9b41-86521e23ffa5 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:36.614027 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e21db1e-09fa-40c5-99b0-3c2da1ef4b8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:36.614732 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e21db1e-09fa-40c5-99b0-3c2da1ef4b8b None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:36.615024 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e21db1e-09fa-40c5-99b0-3c2da1ef4b8b None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:36.615350 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e21db1e-09fa-40c5-99b0-3c2da1ef4b8b None admin] PATH_INFO: `/v3/roles` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:36.616001 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e21db1e-09fa-40c5-99b0-3c2da1ef4b8b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:45:36.616311 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e21db1e-09fa-40c5-99b0-3c2da1ef4b8b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:36.617782 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e21db1e-09fa-40c5-99b0-3c2da1ef4b8b None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:36.630429 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 37/78] 10.222.0.46 () {56 vars in 1111 bytes} [Mon Oct 10 11:45:36 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:45:36.631282 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e21db1e-09fa-40c5-99b0-3c2da1ef4b8b None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:36.676540 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:36.695486 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1120e880-8ae1-4aa7-9b41-86521e23ffa5 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:36.729173 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c940b872-bbe5-416d-8786-dd177f89d06a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:36.729733 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c940b872-bbe5-416d-8786-dd177f89d06a None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:36.730015 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c940b872-bbe5-416d-8786-dd177f89d06a None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:36.730285 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c940b872-bbe5-416d-8786-dd177f89d06a None admin] PATH_INFO: `/v3/domains/default` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:36.731421 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c940b872-bbe5-416d-8786-dd177f89d06a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:36.731602 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e21db1e-09fa-40c5-99b0-3c2da1ef4b8b None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:36.732877 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c940b872-bbe5-416d-8786-dd177f89d06a None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:36.734432 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 42/79] 10.222.0.46 () {56 vars in 1108 bytes} [Mon Oct 10 11:45:36 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:45:36.735292 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c940b872-bbe5-416d-8786-dd177f89d06a None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:36.766983 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65319c6f-d382-4864-bcb2-3b36b38d7b0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:36.767621 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65319c6f-d382-4864-bcb2-3b36b38d7b0a None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:36.767909 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65319c6f-d382-4864-bcb2-3b36b38d7b0a None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:36.768183 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65319c6f-d382-4864-bcb2-3b36b38d7b0a None admin] PATH_INFO: `/v3/users/531d202e4dea4ee38988bfd8b9e26751` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:36.781604 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65319c6f-d382-4864-bcb2-3b36b38d7b0a None admin] RBAC: Authorizing `identity:get_user(user_id=531d202e4dea4ee38988bfd8b9e26751)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:36.783119 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65319c6f-d382-4864-bcb2-3b36b38d7b0a None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:36.785304 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 38/80] 10.222.0.46 () {56 vars in 1177 bytes} [Mon Oct 10 11:45:36 2022] GET /identity/v3/users/531d202e4dea4ee38988bfd8b9e26751 => generated 275 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:45:36.786070 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65319c6f-d382-4864-bcb2-3b36b38d7b0a None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:36.819864 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:36.840905 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c940b872-bbe5-416d-8786-dd177f89d06a None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:36.875323 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01f7626-741f-496d-9a6d-06b8a33f8318 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:36.875939 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01f7626-741f-496d-9a6d-06b8a33f8318 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:36.876225 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01f7626-741f-496d-9a6d-06b8a33f8318 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:36.876532 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01f7626-741f-496d-9a6d-06b8a33f8318 None admin] PATH_INFO: `/v3/roles` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:36.877204 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01f7626-741f-496d-9a6d-06b8a33f8318 None admin] RBAC: Adding query filter params (name=member) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:45:36.877509 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01f7626-741f-496d-9a6d-06b8a33f8318 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:36.879539 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01f7626-741f-496d-9a6d-06b8a33f8318 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:36.887649 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 43/81] 10.222.0.46 () {56 vars in 1101 bytes} [Mon Oct 10 11:45:36 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:45:36.892401 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65319c6f-d382-4864-bcb2-3b36b38d7b0a None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:36.892492 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01f7626-741f-496d-9a6d-06b8a33f8318 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:36.902040 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:36.921868 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b523deb4-92c0-4e79-97ac-9d26ca11be8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:36.922499 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b523deb4-92c0-4e79-97ac-9d26ca11be8b None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:36.922819 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b523deb4-92c0-4e79-97ac-9d26ca11be8b None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:36.923094 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b523deb4-92c0-4e79-97ac-9d26ca11be8b None admin] PATH_INFO: `/v3/users/8cbbc9233ced43d487079e9479dc555c` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:36.924220 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b523deb4-92c0-4e79-97ac-9d26ca11be8b None admin] RBAC: Authorizing `identity:get_user(user_id=8cbbc9233ced43d487079e9479dc555c)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:36.926224 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b523deb4-92c0-4e79-97ac-9d26ca11be8b None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:36.927993 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 39/82] 10.222.0.46 () {56 vars in 1177 bytes} [Mon Oct 10 11:45:36 2022] GET /identity/v3/users/8cbbc9233ced43d487079e9479dc555c => generated 247 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:45:36.928565 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b523deb4-92c0-4e79-97ac-9d26ca11be8b None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:36.991801 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01f7626-741f-496d-9a6d-06b8a33f8318 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:37.024950 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:37.026634 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95be409c-93fc-4c9d-b199-967881e2c0fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:37.027296 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95be409c-93fc-4c9d-b199-967881e2c0fb None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:37.027578 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95be409c-93fc-4c9d-b199-967881e2c0fb None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:37.027849 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95be409c-93fc-4c9d-b199-967881e2c0fb None admin] PATH_INFO: `/v3/users/531d202e4dea4ee38988bfd8b9e26751` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:37.028038 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b523deb4-92c0-4e79-97ac-9d26ca11be8b None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:37.028988 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95be409c-93fc-4c9d-b199-967881e2c0fb None admin] RBAC: Authorizing `identity:get_user(user_id=531d202e4dea4ee38988bfd8b9e26751)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:37.030427 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95be409c-93fc-4c9d-b199-967881e2c0fb None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:37.032030 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 44/83] 10.222.0.46 () {56 vars in 1177 bytes} [Mon Oct 10 11:45:36 2022] GET /identity/v3/users/531d202e4dea4ee38988bfd8b9e26751 => generated 275 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:45:37.032792 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95be409c-93fc-4c9d-b199-967881e2c0fb None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:37.063393 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0525d7-70fb-4438-b808-059edd4a3e4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:37.064185 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0525d7-70fb-4438-b808-059edd4a3e4b None admin] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:37.064464 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0525d7-70fb-4438-b808-059edd4a3e4b None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:37.064744 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0525d7-70fb-4438-b808-059edd4a3e4b None admin] PATH_INFO: `/v3/users` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:37.065422 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0525d7-70fb-4438-b808-059edd4a3e4b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:37.067015 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0525d7-70fb-4438-b808-059edd4a3e4b None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:37.110967 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:37.131897 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95be409c-93fc-4c9d-b199-967881e2c0fb None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:37.149725 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 40/84] 10.222.0.46 () {60 vars in 1132 bytes} [Mon Oct 10 11:45:36 2022] POST /identity/v3/users => generated 289 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 11:45:37.150567 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0525d7-70fb-4438-b808-059edd4a3e4b None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:37.166108 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2015659-4c77-4fe6-8546-9e1641ef6ffd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:37.167039 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2015659-4c77-4fe6-8546-9e1641ef6ffd None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:37.167478 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2015659-4c77-4fe6-8546-9e1641ef6ffd None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:37.169468 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2015659-4c77-4fe6-8546-9e1641ef6ffd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:37.246950 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2015659-4c77-4fe6-8546-9e1641ef6ffd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:37.248390 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2015659-4c77-4fe6-8546-9e1641ef6ffd None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:37.271396 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0525d7-70fb-4438-b808-059edd4a3e4b None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:37.308043 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3b851c-dd96-486b-a5f0-b7fbe25043fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:37.308714 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3b851c-dd96-486b-a5f0-b7fbe25043fb None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:37.309000 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3b851c-dd96-486b-a5f0-b7fbe25043fb None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:37.309280 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3b851c-dd96-486b-a5f0-b7fbe25043fb None admin] PATH_INFO: `/v3/users/531d202e4dea4ee38988bfd8b9e26751` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:37.310486 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3b851c-dd96-486b-a5f0-b7fbe25043fb None admin] RBAC: Authorizing `identity:get_user(user_id=531d202e4dea4ee38988bfd8b9e26751)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:37.311990 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3b851c-dd96-486b-a5f0-b7fbe25043fb None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:37.313772 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 41/85] 10.222.0.46 () {56 vars in 1177 bytes} [Mon Oct 10 11:45:37 2022] GET /identity/v3/users/531d202e4dea4ee38988bfd8b9e26751 => generated 275 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:45:37.314395 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3b851c-dd96-486b-a5f0-b7fbe25043fb None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:37.346172 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:37.348238 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:37.373305 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 45/86] 10.222.0.46 () {58 vars in 1282 bytes} [Mon Oct 10 11:45:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 10 11:45:37.374285 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2015659-4c77-4fe6-8546-9e1641ef6ffd None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:37.442631 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3b851c-dd96-486b-a5f0-b7fbe25043fb None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:37.491213 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b88154-464b-4653-970a-0f792e1a1320 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:37.491909 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b88154-464b-4653-970a-0f792e1a1320 None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:37.492122 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b88154-464b-4653-970a-0f792e1a1320 None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:37.492641 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b88154-464b-4653-970a-0f792e1a1320 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:37.503471 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2015659-4c77-4fe6-8546-9e1641ef6ffd None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:37.538977 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67497d83-8cbc-4b59-91ee-b73504bdec76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:37.539596 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67497d83-8cbc-4b59-91ee-b73504bdec76 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:37.539881 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67497d83-8cbc-4b59-91ee-b73504bdec76 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:37.540159 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67497d83-8cbc-4b59-91ee-b73504bdec76 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:37.583202 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b88154-464b-4653-970a-0f792e1a1320 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:37.584689 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b88154-464b-4653-970a-0f792e1a1320 None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:37.596014 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:37.616826 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:37.621280 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67497d83-8cbc-4b59-91ee-b73504bdec76 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:37.624297 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67497d83-8cbc-4b59-91ee-b73504bdec76 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:37.719516 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 42/87] 10.222.0.46 () {58 vars in 1282 bytes} [Mon Oct 10 11:45:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 10 11:45:37.720550 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b88154-464b-4653-970a-0f792e1a1320 None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:37.745362 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 46/88] 10.222.0.46 () {58 vars in 1282 bytes} [Mon Oct 10 11:45:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 10 11:45:37.746226 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67497d83-8cbc-4b59-91ee-b73504bdec76 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:37.839669 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:37.864302 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67497d83-8cbc-4b59-91ee-b73504bdec76 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:37.869040 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b88154-464b-4653-970a-0f792e1a1320 None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:37.898776 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:37.909988 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb516e0-cad9-43e3-bb94-2092b231105e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:37.910636 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb516e0-cad9-43e3-bb94-2092b231105e None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:37.910944 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb516e0-cad9-43e3-bb94-2092b231105e None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:37.911225 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb516e0-cad9-43e3-bb94-2092b231105e None admin] PATH_INFO: `/v3/projects/2d0af77a941540ef9b78b89ebae1ef1a` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:37.912783 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb516e0-cad9-43e3-bb94-2092b231105e None admin] RBAC: Authorizing `identity:get_project(project_id=2d0af77a941540ef9b78b89ebae1ef1a)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:37.914242 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb516e0-cad9-43e3-bb94-2092b231105e None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:37.916176 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 47/89] 10.222.0.46 () {56 vars in 1186 bytes} [Mon Oct 10 11:45:37 2022] GET /identity/v3/projects/2d0af77a941540ef9b78b89ebae1ef1a => generated 298 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:45:37.918754 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01ced1d-e5f7-4001-851e-ad62b67d35fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:37.919605 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01ced1d-e5f7-4001-851e-ad62b67d35fd None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:37.920118 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01ced1d-e5f7-4001-851e-ad62b67d35fd None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:37.920585 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01ced1d-e5f7-4001-851e-ad62b67d35fd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:37.921649 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb516e0-cad9-43e3-bb94-2092b231105e None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:37.999853 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01ced1d-e5f7-4001-851e-ad62b67d35fd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:38.001388 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01ced1d-e5f7-4001-851e-ad62b67d35fd None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:38.059348 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb516e0-cad9-43e3-bb94-2092b231105e None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:38.110541 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef3d013-0776-4fcc-bf6c-203d4211f155 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:38.111581 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef3d013-0776-4fcc-bf6c-203d4211f155 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:38.112623 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef3d013-0776-4fcc-bf6c-203d4211f155 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:38.112724 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 43/90] 10.222.0.46 () {58 vars in 1282 bytes} [Mon Oct 10 11:45:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 10 11:45:38.113180 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef3d013-0776-4fcc-bf6c-203d4211f155 None admin] PATH_INFO: `/v3/projects/2d0af77a941540ef9b78b89ebae1ef1a` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:38.114519 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef3d013-0776-4fcc-bf6c-203d4211f155 None admin] RBAC: Authorizing `identity:get_project(project_id=2d0af77a941540ef9b78b89ebae1ef1a)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:38.116182 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef3d013-0776-4fcc-bf6c-203d4211f155 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:38.118235 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01ced1d-e5f7-4001-851e-ad62b67d35fd None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:38.118451 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 48/91] 10.222.0.46 () {56 vars in 1186 bytes} [Mon Oct 10 11:45:37 2022] GET /identity/v3/projects/2d0af77a941540ef9b78b89ebae1ef1a => generated 298 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:45:38.119403 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef3d013-0776-4fcc-bf6c-203d4211f155 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:38.139999 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:38.154656 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:38.242366 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef3d013-0776-4fcc-bf6c-203d4211f155 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:38.257119 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01ced1d-e5f7-4001-851e-ad62b67d35fd None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:38.292746 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1718d51c-1d42-4952-a16c-3bb0d14c9f03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:38.293730 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1718d51c-1d42-4952-a16c-3bb0d14c9f03 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:38.293881 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8fb4b9-8c38-44c8-8035-e2913fbd3410 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:38.294365 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1718d51c-1d42-4952-a16c-3bb0d14c9f03 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:38.294741 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8fb4b9-8c38-44c8-8035-e2913fbd3410 None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:38.295015 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1718d51c-1d42-4952-a16c-3bb0d14c9f03 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:38.295110 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8fb4b9-8c38-44c8-8035-e2913fbd3410 None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:38.295456 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8fb4b9-8c38-44c8-8035-e2913fbd3410 None admin] PATH_INFO: `/v3/projects/2d0af77a941540ef9b78b89ebae1ef1a` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:38.296292 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1718d51c-1d42-4952-a16c-3bb0d14c9f03 None admin] RBAC: Adding query filter params (role.id=b2cb2a6e818e47338ce071e6fedfc673, scope.project.id=2d0af77a941540ef9b78b89ebae1ef1a, user.id=531d202e4dea4ee38988bfd8b9e26751) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:45:38.296746 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8fb4b9-8c38-44c8-8035-e2913fbd3410 None admin] RBAC: Authorizing `identity:get_project(project_id=2d0af77a941540ef9b78b89ebae1ef1a)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:38.296961 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1718d51c-1d42-4952-a16c-3bb0d14c9f03 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:38.298532 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1718d51c-1d42-4952-a16c-3bb0d14c9f03 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:38.298630 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8fb4b9-8c38-44c8-8035-e2913fbd3410 None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:38.300417 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 44/92] 10.222.0.46 () {56 vars in 1186 bytes} [Mon Oct 10 11:45:38 2022] GET /identity/v3/projects/2d0af77a941540ef9b78b89ebae1ef1a => generated 298 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:45:38.301264 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8fb4b9-8c38-44c8-8035-e2913fbd3410 None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:38.306586 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 49/93] 10.222.0.46 () {56 vars in 1374 bytes} [Mon Oct 10 11:45:38 2022] GET /identity/v3/role_assignments?user.id=531d202e4dea4ee38988bfd8b9e26751&scope.project.id=2d0af77a941540ef9b78b89ebae1ef1a&role.id=b2cb2a6e818e47338ce071e6fedfc673 => generated 259 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:45:38.307640 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1718d51c-1d42-4952-a16c-3bb0d14c9f03 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:38.406332 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:38.432922 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1718d51c-1d42-4952-a16c-3bb0d14c9f03 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:38.486907 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6dfdd37-3917-48b0-9067-42033478823c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:38.487874 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6dfdd37-3917-48b0-9067-42033478823c None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:38.488384 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6dfdd37-3917-48b0-9067-42033478823c None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:38.488926 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6dfdd37-3917-48b0-9067-42033478823c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:38.489854 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6dfdd37-3917-48b0-9067-42033478823c None admin] RBAC: Adding query filter params (role.id=0d822a0674b6495fac623877a3b38025, scope.project.id=2d0af77a941540ef9b78b89ebae1ef1a, user.id=8cbbc9233ced43d487079e9479dc555c) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:45:38.490347 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6dfdd37-3917-48b0-9067-42033478823c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:38.492216 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6dfdd37-3917-48b0-9067-42033478823c None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:38.500682 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 50/94] 10.222.0.46 () {56 vars in 1374 bytes} [Mon Oct 10 11:45:38 2022] GET /identity/v3/role_assignments?user.id=8cbbc9233ced43d487079e9479dc555c&scope.project.id=2d0af77a941540ef9b78b89ebae1ef1a&role.id=0d822a0674b6495fac623877a3b38025 => generated 259 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:45:38.502082 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6dfdd37-3917-48b0-9067-42033478823c None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:38.546056 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:38.579904 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8fb4b9-8c38-44c8-8035-e2913fbd3410 None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:38.650237 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6dfdd37-3917-48b0-9067-42033478823c None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:38.662087 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1e18ec-3bd7-40ff-9f2d-05979a11f518 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:38.662819 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1e18ec-3bd7-40ff-9f2d-05979a11f518 None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:38.663186 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1e18ec-3bd7-40ff-9f2d-05979a11f518 None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:38.663549 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1e18ec-3bd7-40ff-9f2d-05979a11f518 None admin] PATH_INFO: `/v3/projects/7c32e0f418e949da8cd9bbe662b13191` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:38.664925 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1e18ec-3bd7-40ff-9f2d-05979a11f518 None admin] RBAC: Authorizing `identity:get_project(project_id=7c32e0f418e949da8cd9bbe662b13191)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:38.666584 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1e18ec-3bd7-40ff-9f2d-05979a11f518 None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:38.668588 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 45/95] 10.222.0.46 () {56 vars in 1186 bytes} [Mon Oct 10 11:45:38 2022] GET /identity/v3/projects/7c32e0f418e949da8cd9bbe662b13191 => generated 312 bytes in 368 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:45:38.676559 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1e18ec-3bd7-40ff-9f2d-05979a11f518 None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:38.700342 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10cec85f-efe7-4b2a-a2b1-ceeee0cfe59c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:38.701296 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cec85f-efe7-4b2a-a2b1-ceeee0cfe59c None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:38.702074 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cec85f-efe7-4b2a-a2b1-ceeee0cfe59c None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:38.702567 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cec85f-efe7-4b2a-a2b1-ceeee0cfe59c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:38.703547 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10cec85f-efe7-4b2a-a2b1-ceeee0cfe59c None admin] RBAC: Adding query filter params (role.id=6762beebc2bf4e6f804cbbe55a475e70, scope.project.id=2d0af77a941540ef9b78b89ebae1ef1a, user.id=531d202e4dea4ee38988bfd8b9e26751) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:45:38.704583 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10cec85f-efe7-4b2a-a2b1-ceeee0cfe59c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:38.706326 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10cec85f-efe7-4b2a-a2b1-ceeee0cfe59c None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:38.715037 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 51/96] 10.222.0.46 () {56 vars in 1374 bytes} [Mon Oct 10 11:45:38 2022] GET /identity/v3/role_assignments?user.id=531d202e4dea4ee38988bfd8b9e26751&scope.project.id=2d0af77a941540ef9b78b89ebae1ef1a&role.id=6762beebc2bf4e6f804cbbe55a475e70 => generated 259 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:45:38.823209 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1e18ec-3bd7-40ff-9f2d-05979a11f518 None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:38.863671 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3a4243-b43d-4b22-9cb1-5c6e618610c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:38.864312 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3a4243-b43d-4b22-9cb1-5c6e618610c7 None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:38.864648 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3a4243-b43d-4b22-9cb1-5c6e618610c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:38.864936 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3a4243-b43d-4b22-9cb1-5c6e618610c7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:38.865777 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3a4243-b43d-4b22-9cb1-5c6e618610c7 None admin] RBAC: Adding query filter params (role.id=b2cb2a6e818e47338ce071e6fedfc673, scope.project.id=7c32e0f418e949da8cd9bbe662b13191, user.id=531d202e4dea4ee38988bfd8b9e26751) {{(pid=81676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:45:38.866106 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3a4243-b43d-4b22-9cb1-5c6e618610c7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:38.867688 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3a4243-b43d-4b22-9cb1-5c6e618610c7 None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:38.874598 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 46/97] 10.222.0.46 () {56 vars in 1374 bytes} [Mon Oct 10 11:45:38 2022] GET /identity/v3/role_assignments?user.id=531d202e4dea4ee38988bfd8b9e26751&scope.project.id=7c32e0f418e949da8cd9bbe662b13191&role.id=b2cb2a6e818e47338ce071e6fedfc673 => generated 259 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:45:39.573378 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86462097-88e0-4116-91d9-cf340d533fe8 None None] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:39.573378 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86462097-88e0-4116-91d9-cf340d533fe8 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:39.573378 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86462097-88e0-4116-91d9-cf340d533fe8 None None] PATH_INFO: `/` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:39.573378 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 52/98] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 10 11:45:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:45:39.579052 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f59f74a-f27e-4638-a9fa-6715c6a9fa08 None None] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:39.579547 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f59f74a-f27e-4638-a9fa-6715c6a9fa08 None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:39.579620 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f59f74a-f27e-4638-a9fa-6715c6a9fa08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:39.603908 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.auth.core [None req-1f59f74a-f27e-4638-a9fa-6715c6a9fa08 None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:45:39.605151 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-1f59f74a-f27e-4638-a9fa-6715c6a9fa08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:45:39.627799 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:39.684920 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 47/99] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 10 11:45:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 10 11:45:39.973603 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9591487a-59db-4111-92cd-af03e804a16f None None] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:39.974603 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9591487a-59db-4111-92cd-af03e804a16f None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:39.975126 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9591487a-59db-4111-92cd-af03e804a16f None None] PATH_INFO: `/` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:39.976666 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 53/100] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 10 11:45:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:45:39.982242 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c0ddd2-9bf2-4c07-9436-3edbeb30097a None None] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:39.982539 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c0ddd2-9bf2-4c07-9436-3edbeb30097a None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:39.982867 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c0ddd2-9bf2-4c07-9436-3edbeb30097a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:40.006457 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.auth.core [None req-56c0ddd2-9bf2-4c07-9436-3edbeb30097a None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:45:40.007742 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-56c0ddd2-9bf2-4c07-9436-3edbeb30097a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:45:40.100849 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 48/101] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 10 11:45:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 10 11:45:40.114966 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9591487a-59db-4111-92cd-af03e804a16f None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:40.138720 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:40.284118 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9591487a-59db-4111-92cd-af03e804a16f None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:40.330135 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985f6f73-000b-4f27-93a4-ebf7b9d0366c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:40.331764 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985f6f73-000b-4f27-93a4-ebf7b9d0366c None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:40.332279 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985f6f73-000b-4f27-93a4-ebf7b9d0366c None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:40.332746 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985f6f73-000b-4f27-93a4-ebf7b9d0366c None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:40.350345 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985f6f73-000b-4f27-93a4-ebf7b9d0366c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 10 11:45:40.356417 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application [None req-985f6f73-000b-4f27-93a4-ebf7b9d0366c None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 10 11:45:40.356417 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:45:40.356417 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:45:40.356417 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:45:40.356417 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:45:40.356417 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:45:40.356417 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:45:40.356417 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:45:40.356417 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:45:40.356417 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:45:40.356417 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:45:40.356417 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:45:40.356417 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 10 11:45:40.356417 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 10 11:45:40.356417 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 10 11:45:40.356417 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 10 11:45:40.357318 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 10 11:45:40.357318 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 10 11:45:40.357318 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 10 11:45:40.357318 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 10 11:45:40.357318 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:45:40.357318 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:45:40.357318 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:45:40.357318 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:45:40.357318 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:45:40.357318 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:45:40.357318 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:45:40.357318 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application with Lock( Oct 10 11:45:40.357318 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:45:40.357318 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:45:40.357318 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:45:40.357318 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:45:40.358050 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:45:40.358050 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:45:40.358050 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:45:40.358050 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:45:40.358050 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 10 11:45:40.358050 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 10 11:45:40.358050 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 10 11:45:40.358050 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 10 11:45:40.358050 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 10 11:45:40.358050 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 10 11:45:40.358050 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 10 11:45:40.358050 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Oct 10 11:45:40.360869 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 54/102] 10.222.0.46 () {56 vars in 1099 bytes} [Mon Oct 10 11:45:40 2022] GET /identity/v3/roles/reader => generated 84 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:45:40.366582 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c0ddd2-9bf2-4c07-9436-3edbeb30097a None None] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:40.514349 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:40.538420 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c0ddd2-9bf2-4c07-9436-3edbeb30097a None None] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:40.568725 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985f6f73-000b-4f27-93a4-ebf7b9d0366c None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:40.596068 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e9dd2f-c610-4423-87b7-85f4fab0a33a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:40.596746 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e9dd2f-c610-4423-87b7-85f4fab0a33a None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:40.597007 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e9dd2f-c610-4423-87b7-85f4fab0a33a None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:40.597295 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e9dd2f-c610-4423-87b7-85f4fab0a33a None admin] PATH_INFO: `/v3/roles` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:40.597978 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e9dd2f-c610-4423-87b7-85f4fab0a33a None admin] RBAC: Adding query filter params (name=reader) {{(pid=81676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:45:40.598286 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e9dd2f-c610-4423-87b7-85f4fab0a33a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:40.599728 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e9dd2f-c610-4423-87b7-85f4fab0a33a None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:40.617938 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 49/103] 10.222.0.46 () {56 vars in 1101 bytes} [Mon Oct 10 11:45:40 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:45:40.624005 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e9dd2f-c610-4423-87b7-85f4fab0a33a None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:40.691090 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:40.739263 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985f6f73-000b-4f27-93a4-ebf7b9d0366c None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:40.813550 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bba1e8-bf89-40e2-96db-a283ab1b5573 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:40.814626 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bba1e8-bf89-40e2-96db-a283ab1b5573 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:40.815266 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bba1e8-bf89-40e2-96db-a283ab1b5573 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:40.815786 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bba1e8-bf89-40e2-96db-a283ab1b5573 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:40.817393 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29bba1e8-bf89-40e2-96db-a283ab1b5573 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:40.819312 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29bba1e8-bf89-40e2-96db-a283ab1b5573 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:40.821491 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 55/104] 10.222.0.46 () {56 vars in 1108 bytes} [Mon Oct 10 11:45:40 2022] GET /identity/v3/domains/default => generated 199 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:45:40.827157 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bba1e8-bf89-40e2-96db-a283ab1b5573 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:40.895196 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e9dd2f-c610-4423-87b7-85f4fab0a33a None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:40.995728 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b83a7f1-5b5b-408c-8024-815a416f2b1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:40.996406 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b83a7f1-5b5b-408c-8024-815a416f2b1e None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:40.996675 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b83a7f1-5b5b-408c-8024-815a416f2b1e None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:40.996967 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b83a7f1-5b5b-408c-8024-815a416f2b1e None admin] PATH_INFO: `/v3/users/ae40cb146dee491b8706cc0d8a34df65` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:41.016766 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bba1e8-bf89-40e2-96db-a283ab1b5573 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:41.036000 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b83a7f1-5b5b-408c-8024-815a416f2b1e None admin] RBAC: Authorizing `identity:get_user(user_id=ae40cb146dee491b8706cc0d8a34df65)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:41.037499 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b83a7f1-5b5b-408c-8024-815a416f2b1e None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:41.039269 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 50/105] 10.222.0.46 () {56 vars in 1177 bytes} [Mon Oct 10 11:45:40 2022] GET /identity/v3/users/ae40cb146dee491b8706cc0d8a34df65 => generated 289 bytes in 416 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 11:45:41.047146 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b83a7f1-5b5b-408c-8024-815a416f2b1e None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:41.055013 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:41.085337 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a030161-ffc8-4799-a210-432f626530cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:41.085889 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a030161-ffc8-4799-a210-432f626530cf None admin] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:41.086185 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a030161-ffc8-4799-a210-432f626530cf None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:41.086449 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a030161-ffc8-4799-a210-432f626530cf None admin] PATH_INFO: `/v3/projects` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:41.087225 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a030161-ffc8-4799-a210-432f626530cf None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:41.088742 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a030161-ffc8-4799-a210-432f626530cf None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:41.099304 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:41.198514 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 56/106] 10.222.0.46 () {60 vars in 1141 bytes} [Mon Oct 10 11:45:40 2022] POST /identity/v3/projects => generated 302 bytes in 372 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 10 11:45:41.244590 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b83a7f1-5b5b-408c-8024-815a416f2b1e None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:41.291688 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c591a6e-f1d9-42b0-b225-9f560d965537 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:41.292356 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c591a6e-f1d9-42b0-b225-9f560d965537 None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:41.292645 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c591a6e-f1d9-42b0-b225-9f560d965537 None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:41.292924 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c591a6e-f1d9-42b0-b225-9f560d965537 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:41.386683 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c591a6e-f1d9-42b0-b225-9f560d965537 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:41.388203 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c591a6e-f1d9-42b0-b225-9f560d965537 None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:41.391269 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:41.555131 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9788596e-85e4-4d67-9363-dc4b2039f710 None None] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:41.555767 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9788596e-85e4-4d67-9363-dc4b2039f710 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:41.556245 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9788596e-85e4-4d67-9363-dc4b2039f710 None None] PATH_INFO: `/` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:41.556536 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 51/107] 10.222.0.46 () {58 vars in 1282 bytes} [Mon Oct 10 11:45:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 510 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Oct 10 11:45:41.558041 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 57/108] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 10 11:45:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 10 11:45:41.562318 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c591a6e-f1d9-42b0-b225-9f560d965537 None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:41.563825 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f852b3-d859-4284-a639-9b53e0f48757 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:41.564617 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f852b3-d859-4284-a639-9b53e0f48757 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:41.565131 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f852b3-d859-4284-a639-9b53e0f48757 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:41.588644 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-58f852b3-d859-4284-a639-9b53e0f48757 None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:45:41.590239 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-58f852b3-d859-4284-a639-9b53e0f48757 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:45:41.669493 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 58/109] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 10 11:45:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 10 11:45:41.742267 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c591a6e-f1d9-42b0-b225-9f560d965537 None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:41.745194 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:41.790839 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab9fce2-3eaa-485c-bf3c-a13833a295c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:41.791906 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab9fce2-3eaa-485c-bf3c-a13833a295c9 None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:41.792389 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab9fce2-3eaa-485c-bf3c-a13833a295c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:41.792878 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab9fce2-3eaa-485c-bf3c-a13833a295c9 None admin] PATH_INFO: `/v3/projects/2d0af77a941540ef9b78b89ebae1ef1a` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:41.796052 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab9fce2-3eaa-485c-bf3c-a13833a295c9 None admin] RBAC: Authorizing `identity:get_project(project_id=2d0af77a941540ef9b78b89ebae1ef1a)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:41.797719 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab9fce2-3eaa-485c-bf3c-a13833a295c9 None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:41.800536 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 52/110] 10.222.0.46 () {56 vars in 1186 bytes} [Mon Oct 10 11:45:41 2022] GET /identity/v3/projects/2d0af77a941540ef9b78b89ebae1ef1a => generated 298 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:45:41.807137 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f852b3-d859-4284-a639-9b53e0f48757 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:41.928020 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f852b3-d859-4284-a639-9b53e0f48757 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:41.951102 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:41.966325 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067835f9-5cbd-41a2-acc9-40541327cf3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:41.967015 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067835f9-5cbd-41a2-acc9-40541327cf3c None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:41.985969 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067835f9-5cbd-41a2-acc9-40541327cf3c None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:41.985969 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067835f9-5cbd-41a2-acc9-40541327cf3c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:41.985969 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067835f9-5cbd-41a2-acc9-40541327cf3c None admin] RBAC: Adding query filter params (role.id=54c5cfe84bab471db7992eee92787387, scope.project.id=2d0af77a941540ef9b78b89ebae1ef1a, user.id=ae40cb146dee491b8706cc0d8a34df65) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:45:41.985969 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067835f9-5cbd-41a2-acc9-40541327cf3c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:41.985969 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067835f9-5cbd-41a2-acc9-40541327cf3c None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:41.986329 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 59/111] 10.222.0.46 () {56 vars in 1374 bytes} [Mon Oct 10 11:45:41 2022] GET /identity/v3/role_assignments?user.id=ae40cb146dee491b8706cc0d8a34df65&scope.project.id=2d0af77a941540ef9b78b89ebae1ef1a&role.id=54c5cfe84bab471db7992eee92787387 => generated 259 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:45:42.220312 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab9fce2-3eaa-485c-bf3c-a13833a295c9 None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:42.233012 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb07ac1e-d975-4ead-91a6-df1f2e727a2e None None] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:42.233562 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb07ac1e-d975-4ead-91a6-df1f2e727a2e None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:42.233797 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb07ac1e-d975-4ead-91a6-df1f2e727a2e None None] PATH_INFO: `/` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:42.235146 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 60/112] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 10 11:45:42 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:45:42.241162 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a4302d-679e-4e78-aa8f-08d0620c6533 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:42.241449 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a4302d-679e-4e78-aa8f-08d0620c6533 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:42.241731 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a4302d-679e-4e78-aa8f-08d0620c6533 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:42.270557 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-d0a4302d-679e-4e78-aa8f-08d0620c6533 None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:45:42.272057 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-d0a4302d-679e-4e78-aa8f-08d0620c6533 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:45:42.372646 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 61/113] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 10 11:45:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 10 11:45:42.374274 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738d8aa8-4d6d-4bda-ab7b-ba2af3c3cca4 None None] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:42.374516 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738d8aa8-4d6d-4bda-ab7b-ba2af3c3cca4 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:42.374843 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738d8aa8-4d6d-4bda-ab7b-ba2af3c3cca4 None None] PATH_INFO: `/` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:42.376208 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 62/114] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 10 11:45:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:45:42.382516 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cb3595-bf82-4b13-a887-8b84d02b08de None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:42.382833 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cb3595-bf82-4b13-a887-8b84d02b08de None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:42.383115 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cb3595-bf82-4b13-a887-8b84d02b08de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:42.405906 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-27cb3595-bf82-4b13-a887-8b84d02b08de None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:45:42.407156 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-27cb3595-bf82-4b13-a887-8b84d02b08de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:45:42.436957 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab9fce2-3eaa-485c-bf3c-a13833a295c9 None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:42.490078 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 63/115] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 10 11:45:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 10 11:45:42.495831 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fced1f-0f72-4be3-a1f3-0e79fb86d17d None None] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:42.496137 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fced1f-0f72-4be3-a1f3-0e79fb86d17d None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:42.496489 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fced1f-0f72-4be3-a1f3-0e79fb86d17d None None] PATH_INFO: `/` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:42.498452 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 64/116] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 10 11:45:42 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:45:42.508010 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0472a5e2-f933-41a2-8038-0dda88f139b7 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:42.508478 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0472a5e2-f933-41a2-8038-0dda88f139b7 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:42.508749 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0472a5e2-f933-41a2-8038-0dda88f139b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:42.532339 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e8000a-ef93-4b3f-b7cf-d4955e575c52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:42.533365 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e8000a-ef93-4b3f-b7cf-d4955e575c52 None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:42.533887 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e8000a-ef93-4b3f-b7cf-d4955e575c52 None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:42.534501 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e8000a-ef93-4b3f-b7cf-d4955e575c52 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:42.541131 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-0472a5e2-f933-41a2-8038-0dda88f139b7 None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:45:42.551093 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-0472a5e2-f933-41a2-8038-0dda88f139b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:45:42.556659 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e8000a-ef93-4b3f-b7cf-d4955e575c52 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 10 11:45:42.562194 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application [None req-d8e8000a-ef93-4b3f-b7cf-d4955e575c52 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 10 11:45:42.562194 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:45:42.562194 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:45:42.562194 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:45:42.562194 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:45:42.562194 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:45:42.562194 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:45:42.562194 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:45:42.562194 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:45:42.562194 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:45:42.562194 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:45:42.562194 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:45:42.562194 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 10 11:45:42.562194 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 10 11:45:42.562194 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 10 11:45:42.562194 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 10 11:45:42.563122 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 10 11:45:42.563122 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 10 11:45:42.563122 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 10 11:45:42.563122 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 10 11:45:42.563122 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:45:42.563122 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:45:42.563122 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:45:42.563122 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:45:42.563122 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:45:42.563122 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:45:42.563122 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:45:42.563122 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application with Lock( Oct 10 11:45:42.563122 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:45:42.563122 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:45:42.563122 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:45:42.563122 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:45:42.563866 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:45:42.563866 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:45:42.563866 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:45:42.563866 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:45:42.563866 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 10 11:45:42.563866 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 10 11:45:42.563866 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 10 11:45:42.563866 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 10 11:45:42.563866 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 10 11:45:42.563866 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 10 11:45:42.563866 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 10 11:45:42.563866 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Oct 10 11:45:42.575142 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 53/117] 10.222.0.46 () {56 vars in 1099 bytes} [Mon Oct 10 11:45:42 2022] GET /identity/v3/roles/member => generated 84 bytes in 355 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:45:42.582418 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e8000a-ef93-4b3f-b7cf-d4955e575c52 None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:42.612161 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:42.665611 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:42.684697 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 65/118] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 10 11:45:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 10 11:45:42.736007 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0472a5e2-f933-41a2-8038-0dda88f139b7 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:42.760747 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e8000a-ef93-4b3f-b7cf-d4955e575c52 None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:42.814248 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba37032-e0ed-44f0-9250-38b8a49754b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:42.815449 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba37032-e0ed-44f0-9250-38b8a49754b7 None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:42.815743 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba37032-e0ed-44f0-9250-38b8a49754b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:42.816029 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba37032-e0ed-44f0-9250-38b8a49754b7 None admin] PATH_INFO: `/v3/roles` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:42.816715 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba37032-e0ed-44f0-9250-38b8a49754b7 None admin] RBAC: Adding query filter params (name=member) {{(pid=81676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:45:42.817027 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba37032-e0ed-44f0-9250-38b8a49754b7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:42.818441 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba37032-e0ed-44f0-9250-38b8a49754b7 None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:42.837133 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 54/119] 10.222.0.46 () {56 vars in 1101 bytes} [Mon Oct 10 11:45:42 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 11:45:42.843885 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba37032-e0ed-44f0-9250-38b8a49754b7 None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:42.924756 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0472a5e2-f933-41a2-8038-0dda88f139b7 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:42.988967 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:42.989127 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7efb1c6-7f03-4755-80b8-10ffe2e76aef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:42.989769 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7efb1c6-7f03-4755-80b8-10ffe2e76aef None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:42.990053 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7efb1c6-7f03-4755-80b8-10ffe2e76aef None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:42.990327 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7efb1c6-7f03-4755-80b8-10ffe2e76aef None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:43.004677 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba37032-e0ed-44f0-9250-38b8a49754b7 None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:43.007241 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7efb1c6-7f03-4755-80b8-10ffe2e76aef None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 10 11:45:43.012753 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application [None req-d7efb1c6-7f03-4755-80b8-10ffe2e76aef None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 10 11:45:43.012753 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:45:43.012753 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:45:43.012753 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:45:43.012753 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:45:43.012753 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:45:43.012753 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:45:43.012753 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:45:43.012753 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:45:43.012753 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:45:43.012753 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:45:43.012753 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:45:43.012753 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 10 11:45:43.012753 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 10 11:45:43.012753 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 10 11:45:43.012753 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 10 11:45:43.013618 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 10 11:45:43.013618 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 10 11:45:43.013618 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 10 11:45:43.013618 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 10 11:45:43.013618 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:45:43.013618 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:45:43.013618 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:45:43.013618 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:45:43.013618 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:45:43.013618 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:45:43.013618 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:45:43.013618 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application with Lock( Oct 10 11:45:43.013618 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:45:43.013618 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:45:43.013618 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:45:43.013618 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:45:43.014450 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:45:43.014450 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:45:43.014450 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:45:43.014450 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:45:43.014450 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 10 11:45:43.014450 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 10 11:45:43.014450 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 10 11:45:43.014450 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 10 11:45:43.014450 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 10 11:45:43.014450 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 10 11:45:43.014450 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 10 11:45:43.014450 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Oct 10 11:45:43.016932 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 66/120] 10.222.0.46 () {56 vars in 1114 bytes} [Mon Oct 10 11:45:42 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 10 11:45:43.018184 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7efb1c6-7f03-4755-80b8-10ffe2e76aef None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:43.054346 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03eaf956-fd6d-4028-8440-4b23ba8d9299 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:43.055223 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03eaf956-fd6d-4028-8440-4b23ba8d9299 None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:43.055483 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03eaf956-fd6d-4028-8440-4b23ba8d9299 None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:43.055762 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03eaf956-fd6d-4028-8440-4b23ba8d9299 None admin] PATH_INFO: `/v3/users/531d202e4dea4ee38988bfd8b9e26751` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:43.057341 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:43.058395 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03eaf956-fd6d-4028-8440-4b23ba8d9299 None admin] RBAC: Authorizing `identity:get_user(user_id=531d202e4dea4ee38988bfd8b9e26751)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:43.060767 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03eaf956-fd6d-4028-8440-4b23ba8d9299 None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:43.063780 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 55/121] 10.222.0.46 () {56 vars in 1177 bytes} [Mon Oct 10 11:45:42 2022] GET /identity/v3/users/531d202e4dea4ee38988bfd8b9e26751 => generated 275 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:45:43.064751 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03eaf956-fd6d-4028-8440-4b23ba8d9299 None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:43.140003 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7efb1c6-7f03-4755-80b8-10ffe2e76aef None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:43.184813 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856f8f87-cfd1-4758-bc2d-90ccfe401148 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:43.185468 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856f8f87-cfd1-4758-bc2d-90ccfe401148 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:43.185756 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856f8f87-cfd1-4758-bc2d-90ccfe401148 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:43.186039 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856f8f87-cfd1-4758-bc2d-90ccfe401148 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:43.198365 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03eaf956-fd6d-4028-8440-4b23ba8d9299 None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:43.198525 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856f8f87-cfd1-4758-bc2d-90ccfe401148 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 10 11:45:43.204858 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application [None req-856f8f87-cfd1-4758-bc2d-90ccfe401148 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 10 11:45:43.204858 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:45:43.204858 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:45:43.204858 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:45:43.204858 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:45:43.204858 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:45:43.204858 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:45:43.204858 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:45:43.204858 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:45:43.204858 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:45:43.204858 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:45:43.204858 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:45:43.204858 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 10 11:45:43.204858 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 10 11:45:43.204858 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 10 11:45:43.204858 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 10 11:45:43.205797 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 10 11:45:43.205797 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 10 11:45:43.205797 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 10 11:45:43.205797 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 10 11:45:43.205797 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:45:43.205797 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:45:43.205797 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:45:43.205797 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:45:43.205797 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:45:43.205797 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:45:43.205797 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:45:43.205797 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application with Lock( Oct 10 11:45:43.205797 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:45:43.205797 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:45:43.205797 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:45:43.205797 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:45:43.206534 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:45:43.206534 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:45:43.206534 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:45:43.206534 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:45:43.206534 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 10 11:45:43.206534 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 10 11:45:43.206534 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 10 11:45:43.206534 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 10 11:45:43.206534 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 10 11:45:43.206534 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 10 11:45:43.206534 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 10 11:45:43.206534 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Oct 10 11:45:43.206534 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 67/122] 10.222.0.46 () {56 vars in 1096 bytes} [Mon Oct 10 11:45:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:45:43.207720 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856f8f87-cfd1-4758-bc2d-90ccfe401148 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:43.240921 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6042f22-1104-4153-8280-5f66995dad4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:43.241481 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6042f22-1104-4153-8280-5f66995dad4e None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:43.241774 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6042f22-1104-4153-8280-5f66995dad4e None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:43.242056 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6042f22-1104-4153-8280-5f66995dad4e None admin] PATH_INFO: `/v3/roles/member` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:43.253733 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6042f22-1104-4153-8280-5f66995dad4e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 10 11:45:43.258634 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application [None req-d6042f22-1104-4153-8280-5f66995dad4e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 10 11:45:43.258634 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:45:43.258634 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:45:43.258634 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:45:43.258634 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:45:43.258634 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:45:43.258634 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:45:43.258634 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:45:43.258634 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:45:43.258634 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:45:43.258634 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:45:43.258634 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:45:43.258634 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 10 11:45:43.258634 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 10 11:45:43.258634 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 10 11:45:43.258634 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 10 11:45:43.259518 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 10 11:45:43.259518 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 10 11:45:43.259518 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 10 11:45:43.259518 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 10 11:45:43.259518 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:45:43.259518 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:45:43.259518 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:45:43.259518 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:45:43.259518 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:45:43.259518 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:45:43.259518 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:45:43.259518 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application with Lock( Oct 10 11:45:43.259518 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:45:43.259518 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:45:43.259518 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:45:43.259518 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:45:43.260274 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:45:43.260274 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:45:43.260274 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:45:43.260274 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:45:43.260274 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 10 11:45:43.260274 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 10 11:45:43.260274 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 10 11:45:43.260274 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 10 11:45:43.260274 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 10 11:45:43.260274 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 10 11:45:43.260274 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 10 11:45:43.260274 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Oct 10 11:45:43.260274 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 56/123] 10.222.0.46 () {56 vars in 1099 bytes} [Mon Oct 10 11:45:43 2022] GET /identity/v3/roles/member => generated 84 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:45:43.260920 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6042f22-1104-4153-8280-5f66995dad4e None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:43.287734 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:43.336741 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:43.351509 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856f8f87-cfd1-4758-bc2d-90ccfe401148 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:43.372106 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6042f22-1104-4153-8280-5f66995dad4e None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:43.399291 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a086d1e1-06ec-4663-bb78-6aaa89be7bd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:43.400465 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a086d1e1-06ec-4663-bb78-6aaa89be7bd5 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:43.401111 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a086d1e1-06ec-4663-bb78-6aaa89be7bd5 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:43.401908 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a086d1e1-06ec-4663-bb78-6aaa89be7bd5 None admin] PATH_INFO: `/v3/roles` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:43.403113 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a086d1e1-06ec-4663-bb78-6aaa89be7bd5 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:45:43.403724 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a086d1e1-06ec-4663-bb78-6aaa89be7bd5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:43.406356 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a086d1e1-06ec-4663-bb78-6aaa89be7bd5 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:43.410461 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8daaf7b9-38b8-4e4f-ba86-b9e0c64cfff0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:43.411158 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8daaf7b9-38b8-4e4f-ba86-b9e0c64cfff0 None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:43.411447 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8daaf7b9-38b8-4e4f-ba86-b9e0c64cfff0 None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:43.411729 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8daaf7b9-38b8-4e4f-ba86-b9e0c64cfff0 None admin] PATH_INFO: `/v3/projects/2d0af77a941540ef9b78b89ebae1ef1a` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:43.412875 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8daaf7b9-38b8-4e4f-ba86-b9e0c64cfff0 None admin] RBAC: Authorizing `identity:get_project(project_id=2d0af77a941540ef9b78b89ebae1ef1a)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:43.414634 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8daaf7b9-38b8-4e4f-ba86-b9e0c64cfff0 None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:43.418093 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 68/125] 10.222.0.46 () {56 vars in 1111 bytes} [Mon Oct 10 11:45:43 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:45:43.419273 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a086d1e1-06ec-4663-bb78-6aaa89be7bd5 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:43.421088 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 57/125] 10.222.0.46 () {56 vars in 1186 bytes} [Mon Oct 10 11:45:43 2022] GET /identity/v3/projects/2d0af77a941540ef9b78b89ebae1ef1a => generated 298 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:45:43.421958 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8daaf7b9-38b8-4e4f-ba86-b9e0c64cfff0 None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:43.528891 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:43.545797 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8daaf7b9-38b8-4e4f-ba86-b9e0c64cfff0 None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:43.553151 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a086d1e1-06ec-4663-bb78-6aaa89be7bd5 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:43.591577 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e241f11-a980-4df2-bdf2-bbc1d471acbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:43.592426 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e241f11-a980-4df2-bdf2-bbc1d471acbe None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:43.592565 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135699dd-ea79-4607-b062-bafa26261b28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:43.592989 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e241f11-a980-4df2-bdf2-bbc1d471acbe None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:43.593402 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135699dd-ea79-4607-b062-bafa26261b28 None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:43.593738 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e241f11-a980-4df2-bdf2-bbc1d471acbe None admin] PATH_INFO: `/v3/roles` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:43.593902 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135699dd-ea79-4607-b062-bafa26261b28 None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:43.594527 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135699dd-ea79-4607-b062-bafa26261b28 None admin] PATH_INFO: `/v3/roles` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:43.595547 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e241f11-a980-4df2-bdf2-bbc1d471acbe None admin] RBAC: Adding query filter params (name=admin) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:45:43.595657 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135699dd-ea79-4607-b062-bafa26261b28 None admin] RBAC: Adding query filter params (name=member) {{(pid=81676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:45:43.596637 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135699dd-ea79-4607-b062-bafa26261b28 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:43.596943 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e241f11-a980-4df2-bdf2-bbc1d471acbe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:43.598683 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135699dd-ea79-4607-b062-bafa26261b28 None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:43.599078 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e241f11-a980-4df2-bdf2-bbc1d471acbe None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:43.608338 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 69/126] 10.222.0.46 () {56 vars in 1099 bytes} [Mon Oct 10 11:45:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:45:43.609196 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 58/127] 10.222.0.46 () {56 vars in 1101 bytes} [Mon Oct 10 11:45:43 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:45:43.609995 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135699dd-ea79-4607-b062-bafa26261b28 None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:43.610184 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e241f11-a980-4df2-bdf2-bbc1d471acbe None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:43.633662 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:43.729926 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e241f11-a980-4df2-bdf2-bbc1d471acbe None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:43.738497 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135699dd-ea79-4607-b062-bafa26261b28 None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:43.767079 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5b873d-7a09-4dba-8a59-ef7e0c68cad4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:43.768514 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5b873d-7a09-4dba-8a59-ef7e0c68cad4 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:43.768969 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5b873d-7a09-4dba-8a59-ef7e0c68cad4 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:43.769390 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5b873d-7a09-4dba-8a59-ef7e0c68cad4 None admin] PATH_INFO: `/v3/users/531d202e4dea4ee38988bfd8b9e26751` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:43.770873 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5b873d-7a09-4dba-8a59-ef7e0c68cad4 None admin] RBAC: Authorizing `identity:get_user(user_id=531d202e4dea4ee38988bfd8b9e26751)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:43.772503 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5b873d-7a09-4dba-8a59-ef7e0c68cad4 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:43.774632 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 70/128] 10.222.0.46 () {56 vars in 1177 bytes} [Mon Oct 10 11:45:43 2022] GET /identity/v3/users/531d202e4dea4ee38988bfd8b9e26751 => generated 275 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:45:43.775747 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5b873d-7a09-4dba-8a59-ef7e0c68cad4 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:43.785362 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484fb432-3fb7-457f-92ad-a4247bf39786 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:43.786059 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484fb432-3fb7-457f-92ad-a4247bf39786 None admin] REQUEST_METHOD: `PUT` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:43.786442 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484fb432-3fb7-457f-92ad-a4247bf39786 None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:43.786851 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484fb432-3fb7-457f-92ad-a4247bf39786 None admin] PATH_INFO: `/v3/projects/2d0af77a941540ef9b78b89ebae1ef1a/users/531d202e4dea4ee38988bfd8b9e26751/roles/b2cb2a6e818e47338ce071e6fedfc673` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:43.789541 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484fb432-3fb7-457f-92ad-a4247bf39786 None admin] RBAC: Authorizing `identity:create_grant(project_id=2d0af77a941540ef9b78b89ebae1ef1a, user_id=531d202e4dea4ee38988bfd8b9e26751, role_id=b2cb2a6e818e47338ce071e6fedfc673)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:43.792029 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484fb432-3fb7-457f-92ad-a4247bf39786 None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:43.855990 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 59/129] 10.222.0.46 () {58 vars in 1439 bytes} [Mon Oct 10 11:45:43 2022] PUT /identity/v3/projects/2d0af77a941540ef9b78b89ebae1ef1a/users/531d202e4dea4ee38988bfd8b9e26751/roles/b2cb2a6e818e47338ce071e6fedfc673 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 11:45:43.857064 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484fb432-3fb7-457f-92ad-a4247bf39786 None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:43.870895 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:43.885171 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:43.923254 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5b873d-7a09-4dba-8a59-ef7e0c68cad4 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:44.004655 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c2b35d-92a5-411d-89aa-27db12f1ad41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:44.007202 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c2b35d-92a5-411d-89aa-27db12f1ad41 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:44.008644 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c2b35d-92a5-411d-89aa-27db12f1ad41 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:44.009182 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c2b35d-92a5-411d-89aa-27db12f1ad41 None admin] PATH_INFO: `/v3/users/8cbbc9233ced43d487079e9479dc555c` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:44.011561 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c2b35d-92a5-411d-89aa-27db12f1ad41 None admin] RBAC: Authorizing `identity:get_user(user_id=8cbbc9233ced43d487079e9479dc555c)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:44.013952 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c2b35d-92a5-411d-89aa-27db12f1ad41 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:44.017247 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 71/130] 10.222.0.46 () {56 vars in 1177 bytes} [Mon Oct 10 11:45:43 2022] GET /identity/v3/users/8cbbc9233ced43d487079e9479dc555c => generated 247 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:45:44.019365 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c2b35d-92a5-411d-89aa-27db12f1ad41 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:44.079747 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484fb432-3fb7-457f-92ad-a4247bf39786 None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:44.133355 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f60fad-6734-452d-9bb6-6c9d1b98a033 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:44.134033 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f60fad-6734-452d-9bb6-6c9d1b98a033 None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:44.134318 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f60fad-6734-452d-9bb6-6c9d1b98a033 None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:44.134599 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f60fad-6734-452d-9bb6-6c9d1b98a033 None admin] PATH_INFO: `/v3/users/531d202e4dea4ee38988bfd8b9e26751` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:44.138997 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f60fad-6734-452d-9bb6-6c9d1b98a033 None admin] RBAC: Authorizing `identity:get_user(user_id=531d202e4dea4ee38988bfd8b9e26751)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:44.140550 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f60fad-6734-452d-9bb6-6c9d1b98a033 None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:44.143633 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 60/131] 10.222.0.46 () {56 vars in 1177 bytes} [Mon Oct 10 11:45:43 2022] GET /identity/v3/users/531d202e4dea4ee38988bfd8b9e26751 => generated 275 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:45:44.144924 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f60fad-6734-452d-9bb6-6c9d1b98a033 None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:44.184137 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c2b35d-92a5-411d-89aa-27db12f1ad41 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:44.230697 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bf4baa-3630-40f1-b93a-71a8f8b7cbf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:44.230697 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bf4baa-3630-40f1-b93a-71a8f8b7cbf6 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:44.230697 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bf4baa-3630-40f1-b93a-71a8f8b7cbf6 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:44.230697 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bf4baa-3630-40f1-b93a-71a8f8b7cbf6 None admin] PATH_INFO: `/v3/projects/2d0af77a941540ef9b78b89ebae1ef1a` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:44.231083 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14bf4baa-3630-40f1-b93a-71a8f8b7cbf6 None admin] RBAC: Authorizing `identity:get_project(project_id=2d0af77a941540ef9b78b89ebae1ef1a)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:44.232100 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14bf4baa-3630-40f1-b93a-71a8f8b7cbf6 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:44.234714 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 72/132] 10.222.0.46 () {56 vars in 1186 bytes} [Mon Oct 10 11:45:44 2022] GET /identity/v3/projects/2d0af77a941540ef9b78b89ebae1ef1a => generated 298 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:45:44.235638 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bf4baa-3630-40f1-b93a-71a8f8b7cbf6 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:44.259711 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:44.320855 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:44.359460 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f60fad-6734-452d-9bb6-6c9d1b98a033 None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:44.360667 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bf4baa-3630-40f1-b93a-71a8f8b7cbf6 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:44.401174 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e6330b-6113-4f47-ba14-056942803792 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:44.401817 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e6330b-6113-4f47-ba14-056942803792 None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:44.402106 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e6330b-6113-4f47-ba14-056942803792 None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:44.402372 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e6330b-6113-4f47-ba14-056942803792 None admin] PATH_INFO: `/v3/projects/2d0af77a941540ef9b78b89ebae1ef1a` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:44.403540 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e6330b-6113-4f47-ba14-056942803792 None admin] RBAC: Authorizing `identity:get_project(project_id=2d0af77a941540ef9b78b89ebae1ef1a)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:44.404562 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c7fd4f-d2b1-46a5-b899-c98ebde16a77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:44.405002 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e6330b-6113-4f47-ba14-056942803792 None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:44.405627 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c7fd4f-d2b1-46a5-b899-c98ebde16a77 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:44.406124 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c7fd4f-d2b1-46a5-b899-c98ebde16a77 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:44.406596 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c7fd4f-d2b1-46a5-b899-c98ebde16a77 None admin] PATH_INFO: `/v3/projects/7c32e0f418e949da8cd9bbe662b13191` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:44.406719 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 61/133] 10.222.0.46 () {56 vars in 1186 bytes} [Mon Oct 10 11:45:44 2022] GET /identity/v3/projects/2d0af77a941540ef9b78b89ebae1ef1a => generated 298 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:45:44.407881 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e6330b-6113-4f47-ba14-056942803792 None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:44.408508 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c7fd4f-d2b1-46a5-b899-c98ebde16a77 None admin] RBAC: Authorizing `identity:get_project(project_id=7c32e0f418e949da8cd9bbe662b13191)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:44.410196 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c7fd4f-d2b1-46a5-b899-c98ebde16a77 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:44.412696 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 73/134] 10.222.0.46 () {56 vars in 1186 bytes} [Mon Oct 10 11:45:44 2022] GET /identity/v3/projects/7c32e0f418e949da8cd9bbe662b13191 => generated 312 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:45:44.414310 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a4b7d6-2b68-45e4-95ff-244a3eaead43 None None] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:44.414826 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a4b7d6-2b68-45e4-95ff-244a3eaead43 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:44.415330 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a4b7d6-2b68-45e4-95ff-244a3eaead43 None None] PATH_INFO: `/` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:44.417048 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 74/135] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 10 11:45:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:45:44.418223 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a4b7d6-2b68-45e4-95ff-244a3eaead43 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:44.527206 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e6330b-6113-4f47-ba14-056942803792 None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:44.544319 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:44.561731 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a4b7d6-2b68-45e4-95ff-244a3eaead43 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:44.566642 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a50c2a2-7fea-4c5b-8875-a76ef4b33c54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:44.567520 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a50c2a2-7fea-4c5b-8875-a76ef4b33c54 None admin] REQUEST_METHOD: `PUT` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:44.567942 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a50c2a2-7fea-4c5b-8875-a76ef4b33c54 None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:44.568354 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a50c2a2-7fea-4c5b-8875-a76ef4b33c54 None admin] PATH_INFO: `/v3/projects/2d0af77a941540ef9b78b89ebae1ef1a/users/531d202e4dea4ee38988bfd8b9e26751/roles/6762beebc2bf4e6f804cbbe55a475e70` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:44.570726 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a50c2a2-7fea-4c5b-8875-a76ef4b33c54 None admin] RBAC: Authorizing `identity:create_grant(project_id=2d0af77a941540ef9b78b89ebae1ef1a, user_id=531d202e4dea4ee38988bfd8b9e26751, role_id=6762beebc2bf4e6f804cbbe55a475e70)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:44.572648 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a50c2a2-7fea-4c5b-8875-a76ef4b33c54 None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:44.602801 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112500c0-4d7b-4166-b7b6-a4b417f100d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:44.603640 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112500c0-4d7b-4166-b7b6-a4b417f100d6 None admin] REQUEST_METHOD: `PUT` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:44.603994 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112500c0-4d7b-4166-b7b6-a4b417f100d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:44.604348 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112500c0-4d7b-4166-b7b6-a4b417f100d6 None admin] PATH_INFO: `/v3/projects/2d0af77a941540ef9b78b89ebae1ef1a/users/8cbbc9233ced43d487079e9479dc555c/roles/0d822a0674b6495fac623877a3b38025` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:44.606343 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112500c0-4d7b-4166-b7b6-a4b417f100d6 None admin] RBAC: Authorizing `identity:create_grant(project_id=2d0af77a941540ef9b78b89ebae1ef1a, user_id=8cbbc9233ced43d487079e9479dc555c, role_id=0d822a0674b6495fac623877a3b38025)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:44.607973 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112500c0-4d7b-4166-b7b6-a4b417f100d6 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:44.675503 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 62/136] 10.222.0.46 () {58 vars in 1439 bytes} [Mon Oct 10 11:45:44 2022] PUT /identity/v3/projects/2d0af77a941540ef9b78b89ebae1ef1a/users/531d202e4dea4ee38988bfd8b9e26751/roles/6762beebc2bf4e6f804cbbe55a475e70 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 11:45:44.682865 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a50c2a2-7fea-4c5b-8875-a76ef4b33c54 None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:44.696143 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:44.704111 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 75/137] 10.222.0.46 () {58 vars in 1439 bytes} [Mon Oct 10 11:45:44 2022] PUT /identity/v3/projects/2d0af77a941540ef9b78b89ebae1ef1a/users/8cbbc9233ced43d487079e9479dc555c/roles/0d822a0674b6495fac623877a3b38025 => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 11:45:44.707002 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b98648-168f-48c1-a85a-e58b22680028 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:44.707002 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b98648-168f-48c1-a85a-e58b22680028 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:44.707002 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b98648-168f-48c1-a85a-e58b22680028 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:44.726793 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-66b98648-168f-48c1-a85a-e58b22680028 None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:45:44.728113 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-66b98648-168f-48c1-a85a-e58b22680028 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:45:44.846628 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 76/138] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 10 11:45:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 10 11:45:44.848384 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a7b1d4-3d7b-4183-a8fd-c785e4256897 None None] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:44.848867 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a7b1d4-3d7b-4183-a8fd-c785e4256897 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:44.849320 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a7b1d4-3d7b-4183-a8fd-c785e4256897 None None] PATH_INFO: `/` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:44.850979 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 77/139] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 10 11:45:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:45:44.861414 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf14268-87bb-4b20-b3a5-0019705a6f5c None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:44.862134 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf14268-87bb-4b20-b3a5-0019705a6f5c None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:44.862592 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf14268-87bb-4b20-b3a5-0019705a6f5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:44.877073 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a50c2a2-7fea-4c5b-8875-a76ef4b33c54 None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:44.902220 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-acf14268-87bb-4b20-b3a5-0019705a6f5c None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:45:44.903974 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-acf14268-87bb-4b20-b3a5-0019705a6f5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:45:44.948651 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3084b03-1080-4826-9af0-859cef470fc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:44.949916 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3084b03-1080-4826-9af0-859cef470fc6 None admin] REQUEST_METHOD: `PUT` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:44.950377 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3084b03-1080-4826-9af0-859cef470fc6 None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:44.951051 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3084b03-1080-4826-9af0-859cef470fc6 None admin] PATH_INFO: `/v3/projects/7c32e0f418e949da8cd9bbe662b13191/users/531d202e4dea4ee38988bfd8b9e26751/roles/b2cb2a6e818e47338ce071e6fedfc673` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:44.953933 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3084b03-1080-4826-9af0-859cef470fc6 None admin] RBAC: Authorizing `identity:create_grant(project_id=7c32e0f418e949da8cd9bbe662b13191, user_id=531d202e4dea4ee38988bfd8b9e26751, role_id=b2cb2a6e818e47338ce071e6fedfc673)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:44.956973 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3084b03-1080-4826-9af0-859cef470fc6 None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:45.012165 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 78/140] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 10 11:45:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 10 11:45:45.042201 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 63/141] 10.222.0.46 () {58 vars in 1439 bytes} [Mon Oct 10 11:45:44 2022] PUT /identity/v3/projects/7c32e0f418e949da8cd9bbe662b13191/users/531d202e4dea4ee38988bfd8b9e26751/roles/b2cb2a6e818e47338ce071e6fedfc673 => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 11:45:45.225275 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf14268-87bb-4b20-b3a5-0019705a6f5c None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:45.234924 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:45.327403 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3084b03-1080-4826-9af0-859cef470fc6 None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:45.356543 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf14268-87bb-4b20-b3a5-0019705a6f5c None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:45.401576 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd172cf7-4b70-4f93-af84-4ebaa9308842 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:45.402217 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd172cf7-4b70-4f93-af84-4ebaa9308842 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:45.402494 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd172cf7-4b70-4f93-af84-4ebaa9308842 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:45.402800 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd172cf7-4b70-4f93-af84-4ebaa9308842 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:45.403938 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd172cf7-4b70-4f93-af84-4ebaa9308842 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:45.405395 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd172cf7-4b70-4f93-af84-4ebaa9308842 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:45.411339 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 79/142] 10.222.0.46 () {56 vars in 1108 bytes} [Mon Oct 10 11:45:45 2022] GET /identity/v3/domains/default => generated 199 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:45:45.414228 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd172cf7-4b70-4f93-af84-4ebaa9308842 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:45.503427 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:45.514437 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:45.549465 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd172cf7-4b70-4f93-af84-4ebaa9308842 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:45.564345 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3084b03-1080-4826-9af0-859cef470fc6 None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:45.588039 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4b3588-d64c-4581-b736-9310d4ede9c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:45.588660 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4b3588-d64c-4581-b736-9310d4ede9c5 None admin] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:45.588938 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4b3588-d64c-4581-b736-9310d4ede9c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:45.589210 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4b3588-d64c-4581-b736-9310d4ede9c5 None admin] PATH_INFO: `/v3/users` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:45.590082 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4b3588-d64c-4581-b736-9310d4ede9c5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:45.591474 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4b3588-d64c-4581-b736-9310d4ede9c5 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:45.611818 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c65d55-477c-44f8-894a-d2c0fba18e6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:45.612577 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c65d55-477c-44f8-894a-d2c0fba18e6b None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:45.612858 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c65d55-477c-44f8-894a-d2c0fba18e6b None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:45.614894 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c65d55-477c-44f8-894a-d2c0fba18e6b None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:45.631559 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c65d55-477c-44f8-894a-d2c0fba18e6b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 10 11:45:45.637160 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application [None req-23c65d55-477c-44f8-894a-d2c0fba18e6b None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 10 11:45:45.637160 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:45:45.637160 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:45:45.637160 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:45:45.637160 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:45:45.637160 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:45:45.637160 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:45:45.637160 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:45:45.637160 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:45:45.637160 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:45:45.637160 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:45:45.637160 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:45:45.637160 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 10 11:45:45.637160 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 10 11:45:45.637160 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 10 11:45:45.637160 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 10 11:45:45.637985 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 10 11:45:45.637985 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 10 11:45:45.637985 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 10 11:45:45.637985 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 10 11:45:45.637985 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:45:45.637985 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:45:45.637985 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:45:45.637985 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:45:45.637985 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:45:45.637985 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:45:45.637985 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:45:45.637985 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application with Lock( Oct 10 11:45:45.637985 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:45:45.637985 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:45:45.637985 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:45:45.637985 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:45:45.638742 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:45:45.638742 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:45:45.638742 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:45:45.638742 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:45:45.638742 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 10 11:45:45.638742 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 10 11:45:45.638742 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 10 11:45:45.638742 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 10 11:45:45.638742 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 10 11:45:45.638742 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 10 11:45:45.638742 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 10 11:45:45.638742 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Oct 10 11:45:45.638742 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 64/143] 10.222.0.46 () {56 vars in 1099 bytes} [Mon Oct 10 11:45:45 2022] GET /identity/v3/roles/reader => generated 84 bytes in 311 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:45:45.647747 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c65d55-477c-44f8-894a-d2c0fba18e6b None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:45.756193 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 80/144] 10.222.0.46 () {60 vars in 1132 bytes} [Mon Oct 10 11:45:45 2022] POST /identity/v3/users => generated 283 bytes in 342 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 11:45:45.773509 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c65d55-477c-44f8-894a-d2c0fba18e6b None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:45.809942 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:45.819489 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aab88d1-25b1-44e4-baf7-f7cca11031a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:45.820181 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aab88d1-25b1-44e4-baf7-f7cca11031a7 None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:45.820472 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aab88d1-25b1-44e4-baf7-f7cca11031a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:45.820752 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aab88d1-25b1-44e4-baf7-f7cca11031a7 None admin] PATH_INFO: `/v3/roles` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:45.821440 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aab88d1-25b1-44e4-baf7-f7cca11031a7 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:45:45.821746 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aab88d1-25b1-44e4-baf7-f7cca11031a7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:45.823218 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aab88d1-25b1-44e4-baf7-f7cca11031a7 None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:45.836992 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 65/145] 10.222.0.46 () {56 vars in 1101 bytes} [Mon Oct 10 11:45:45 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 11:45:45.842977 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4b3588-d64c-4581-b736-9310d4ede9c5 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:45.973066 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4b3588-d64c-4581-b736-9310d4ede9c5 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:46.011990 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:46.019287 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950b2692-76a9-47d1-abee-42585ed100c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:46.020003 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950b2692-76a9-47d1-abee-42585ed100c2 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:46.020283 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950b2692-76a9-47d1-abee-42585ed100c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:46.020556 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950b2692-76a9-47d1-abee-42585ed100c2 None admin] PATH_INFO: `/v3/users/ae40cb146dee491b8706cc0d8a34df65` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:46.022794 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950b2692-76a9-47d1-abee-42585ed100c2 None admin] RBAC: Authorizing `identity:get_user(user_id=ae40cb146dee491b8706cc0d8a34df65)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:46.024320 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950b2692-76a9-47d1-abee-42585ed100c2 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:46.026229 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 81/146] 10.222.0.46 () {56 vars in 1177 bytes} [Mon Oct 10 11:45:45 2022] GET /identity/v3/users/ae40cb146dee491b8706cc0d8a34df65 => generated 289 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:45:46.031856 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aab88d1-25b1-44e4-baf7-f7cca11031a7 None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:46.165393 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aab88d1-25b1-44e4-baf7-f7cca11031a7 None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:46.209780 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f2f10e-cce4-40c3-aabe-696775e24869 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:46.210436 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f2f10e-cce4-40c3-aabe-696775e24869 None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:46.210737 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f2f10e-cce4-40c3-aabe-696775e24869 None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:46.211019 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f2f10e-cce4-40c3-aabe-696775e24869 None admin] PATH_INFO: `/v3/projects/2d0af77a941540ef9b78b89ebae1ef1a` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:46.216121 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f2f10e-cce4-40c3-aabe-696775e24869 None admin] RBAC: Authorizing `identity:get_project(project_id=2d0af77a941540ef9b78b89ebae1ef1a)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:46.217830 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f2f10e-cce4-40c3-aabe-696775e24869 None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:46.220014 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 66/147] 10.222.0.46 () {56 vars in 1186 bytes} [Mon Oct 10 11:45:46 2022] GET /identity/v3/projects/2d0af77a941540ef9b78b89ebae1ef1a => generated 298 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:45:46.226548 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950b2692-76a9-47d1-abee-42585ed100c2 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:46.369892 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950b2692-76a9-47d1-abee-42585ed100c2 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:46.419985 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d5a7ec-c3d5-48d5-9828-ce1e5148b568 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:46.420748 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d5a7ec-c3d5-48d5-9828-ce1e5148b568 None admin] REQUEST_METHOD: `PUT` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:46.420927 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d5a7ec-c3d5-48d5-9828-ce1e5148b568 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:46.421206 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d5a7ec-c3d5-48d5-9828-ce1e5148b568 None admin] PATH_INFO: `/v3/projects/2d0af77a941540ef9b78b89ebae1ef1a/users/ae40cb146dee491b8706cc0d8a34df65/roles/54c5cfe84bab471db7992eee92787387` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:46.428006 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d5a7ec-c3d5-48d5-9828-ce1e5148b568 None admin] RBAC: Authorizing `identity:create_grant(project_id=2d0af77a941540ef9b78b89ebae1ef1a, user_id=ae40cb146dee491b8706cc0d8a34df65, role_id=54c5cfe84bab471db7992eee92787387)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:46.429966 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d5a7ec-c3d5-48d5-9828-ce1e5148b568 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:46.630950 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 82/148] 10.222.0.46 () {58 vars in 1439 bytes} [Mon Oct 10 11:45:46 2022] PUT /identity/v3/projects/2d0af77a941540ef9b78b89ebae1ef1a/users/ae40cb146dee491b8706cc0d8a34df65/roles/54c5cfe84bab471db7992eee92787387 => generated 0 bytes in 405 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 11:45:47.262589 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66f4edd-9744-4586-b597-79328cf42ac0 None None] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:47.263293 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66f4edd-9744-4586-b597-79328cf42ac0 None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:47.263293 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66f4edd-9744-4586-b597-79328cf42ac0 None None] PATH_INFO: `/` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:47.264665 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 67/149] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 10 11:45:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:45:47.282701 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3080158-82fe-4d9f-88f6-60482d57528a None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:47.284050 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3080158-82fe-4d9f-88f6-60482d57528a None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:47.284571 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3080158-82fe-4d9f-88f6-60482d57528a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:47.315735 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-f3080158-82fe-4d9f-88f6-60482d57528a None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:45:47.317322 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-f3080158-82fe-4d9f-88f6-60482d57528a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:45:47.386010 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:47.411058 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 83/150] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 10 11:45:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 10 11:45:47.727454 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66f4edd-9744-4586-b597-79328cf42ac0 None None] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:47.836149 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:47.903848 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66f4edd-9744-4586-b597-79328cf42ac0 None None] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:47.966551 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312cd153-8a33-4c6f-b282-a643ee7912cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:47.967576 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312cd153-8a33-4c6f-b282-a643ee7912cc None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:47.968026 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312cd153-8a33-4c6f-b282-a643ee7912cc None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:47.968461 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312cd153-8a33-4c6f-b282-a643ee7912cc None admin] PATH_INFO: `/v3/roles/member` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:47.971948 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45c143e-2f38-42db-967c-b253ddcbd5b1 None None] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:47.972444 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45c143e-2f38-42db-967c-b253ddcbd5b1 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:47.972848 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45c143e-2f38-42db-967c-b253ddcbd5b1 None None] PATH_INFO: `/` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:47.975268 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 84/151] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 10 11:45:47 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:45:47.982262 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c12d795-5c58-4035-9c63-d1ce35a6c1df None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:47.982616 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c12d795-5c58-4035-9c63-d1ce35a6c1df None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:47.983056 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c12d795-5c58-4035-9c63-d1ce35a6c1df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:48.014597 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312cd153-8a33-4c6f-b282-a643ee7912cc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 10 11:45:48.027909 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application [None req-312cd153-8a33-4c6f-b282-a643ee7912cc None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 10 11:45:48.027909 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:45:48.027909 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:45:48.027909 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:45:48.027909 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:45:48.027909 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:45:48.027909 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:45:48.027909 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:45:48.027909 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:45:48.027909 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:45:48.027909 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:45:48.027909 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:45:48.027909 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 10 11:45:48.027909 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 10 11:45:48.027909 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 10 11:45:48.027909 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 10 11:45:48.028801 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 10 11:45:48.028801 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 10 11:45:48.028801 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 10 11:45:48.028801 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 10 11:45:48.028801 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:45:48.028801 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:45:48.028801 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:45:48.028801 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:45:48.028801 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:45:48.028801 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:45:48.028801 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:45:48.028801 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application with Lock( Oct 10 11:45:48.028801 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:45:48.028801 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:45:48.028801 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:45:48.028801 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:45:48.029549 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:45:48.029549 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:45:48.029549 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:45:48.029549 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:45:48.029549 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 10 11:45:48.029549 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 10 11:45:48.029549 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 10 11:45:48.029549 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 10 11:45:48.029549 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 10 11:45:48.029549 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 10 11:45:48.029549 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 10 11:45:48.029549 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Oct 10 11:45:48.030599 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-7c12d795-5c58-4035-9c63-d1ce35a6c1df None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:45:48.031892 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-7c12d795-5c58-4035-9c63-d1ce35a6c1df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:45:48.032429 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 68/152] 10.222.0.46 () {56 vars in 1099 bytes} [Mon Oct 10 11:45:47 2022] GET /identity/v3/roles/member => generated 84 bytes in 306 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:45:48.058943 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312cd153-8a33-4c6f-b282-a643ee7912cc None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:48.190935 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 85/153] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 10 11:45:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 10 11:45:48.193480 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939100d1-a948-46ef-ac0c-b5a14a6b3a23 None None] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:48.194088 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939100d1-a948-46ef-ac0c-b5a14a6b3a23 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:48.194654 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939100d1-a948-46ef-ac0c-b5a14a6b3a23 None None] PATH_INFO: `/` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:48.197226 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 86/154] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 10 11:45:48 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:45:48.209369 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e892cd95-be3b-438b-b8fb-e9e1883ef961 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:48.209848 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e892cd95-be3b-438b-b8fb-e9e1883ef961 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:48.210327 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e892cd95-be3b-438b-b8fb-e9e1883ef961 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:48.259775 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-e892cd95-be3b-438b-b8fb-e9e1883ef961 None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:45:48.261814 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-e892cd95-be3b-438b-b8fb-e9e1883ef961 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:45:48.285586 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312cd153-8a33-4c6f-b282-a643ee7912cc None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:48.332342 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:48.376393 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a05830-63dc-42cc-8c5a-32f0d07b3b0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:48.378856 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a05830-63dc-42cc-8c5a-32f0d07b3b0e None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:48.379346 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a05830-63dc-42cc-8c5a-32f0d07b3b0e None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:48.379828 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a05830-63dc-42cc-8c5a-32f0d07b3b0e None admin] PATH_INFO: `/v3/roles` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:48.380720 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a05830-63dc-42cc-8c5a-32f0d07b3b0e None admin] RBAC: Adding query filter params (name=member) {{(pid=81676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:45:48.381228 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a05830-63dc-42cc-8c5a-32f0d07b3b0e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:48.382855 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a05830-63dc-42cc-8c5a-32f0d07b3b0e None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:48.390586 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 87/155] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 10 11:45:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 10 11:45:48.393780 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde99a4a-78fc-41cf-b624-7c8033ec46eb None None] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:48.394542 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde99a4a-78fc-41cf-b624-7c8033ec46eb None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:48.395221 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde99a4a-78fc-41cf-b624-7c8033ec46eb None None] PATH_INFO: `/` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:48.396936 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 88/156] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 10 11:45:48 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:45:48.403594 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67226e1-9f4b-4d33-86d7-d04e8a8a68cd None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:48.404203 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67226e1-9f4b-4d33-86d7-d04e8a8a68cd None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:48.404735 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67226e1-9f4b-4d33-86d7-d04e8a8a68cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:48.404924 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 69/157] 10.222.0.46 () {56 vars in 1101 bytes} [Mon Oct 10 11:45:48 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 346 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 11:45:48.410871 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a05830-63dc-42cc-8c5a-32f0d07b3b0e None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:48.442310 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-a67226e1-9f4b-4d33-86d7-d04e8a8a68cd None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:45:48.444252 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-a67226e1-9f4b-4d33-86d7-d04e8a8a68cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:45:48.523494 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:48.551991 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 89/158] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 10 11:45:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 10 11:45:48.636399 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a05830-63dc-42cc-8c5a-32f0d07b3b0e None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:48.682084 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b894ff0d-2c3e-4a57-b140-73cd1aa0672a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:48.682762 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b894ff0d-2c3e-4a57-b140-73cd1aa0672a None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:48.683046 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b894ff0d-2c3e-4a57-b140-73cd1aa0672a None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:48.683321 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b894ff0d-2c3e-4a57-b140-73cd1aa0672a None admin] PATH_INFO: `/v3/users/531d202e4dea4ee38988bfd8b9e26751` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:48.684513 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b894ff0d-2c3e-4a57-b140-73cd1aa0672a None admin] RBAC: Authorizing `identity:get_user(user_id=531d202e4dea4ee38988bfd8b9e26751)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:48.686045 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b894ff0d-2c3e-4a57-b140-73cd1aa0672a None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:48.687743 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 70/159] 10.222.0.46 () {56 vars in 1177 bytes} [Mon Oct 10 11:45:48 2022] GET /identity/v3/users/531d202e4dea4ee38988bfd8b9e26751 => generated 275 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:45:48.694382 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67226e1-9f4b-4d33-86d7-d04e8a8a68cd None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:48.759917 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b894ff0d-2c3e-4a57-b140-73cd1aa0672a None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:48.837088 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67226e1-9f4b-4d33-86d7-d04e8a8a68cd None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:48.876348 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f2138b-b90a-4ad6-bacd-ee7c275934d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:48.878223 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f2138b-b90a-4ad6-bacd-ee7c275934d0 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:48.878780 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f2138b-b90a-4ad6-bacd-ee7c275934d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:48.879269 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f2138b-b90a-4ad6-bacd-ee7c275934d0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:48.879369 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:48.933174 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:48.960437 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f2138b-b90a-4ad6-bacd-ee7c275934d0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:48.962363 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f2138b-b90a-4ad6-bacd-ee7c275934d0 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:49.020403 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b894ff0d-2c3e-4a57-b140-73cd1aa0672a None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:49.127870 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40bbfb1-be47-48df-bc5b-72146cbc29d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:49.128948 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40bbfb1-be47-48df-bc5b-72146cbc29d1 None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:49.129474 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40bbfb1-be47-48df-bc5b-72146cbc29d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:49.129952 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40bbfb1-be47-48df-bc5b-72146cbc29d1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:49.151204 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 90/160] 10.222.0.46 () {58 vars in 1282 bytes} [Mon Oct 10 11:45:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 457 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 10 11:45:49.161818 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40bbfb1-be47-48df-bc5b-72146cbc29d1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 10 11:45:49.174838 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f2138b-b90a-4ad6-bacd-ee7c275934d0 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:49.177322 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application [None req-c40bbfb1-be47-48df-bc5b-72146cbc29d1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 10 11:45:49.177322 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:45:49.177322 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:45:49.177322 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:45:49.177322 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:45:49.177322 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:45:49.177322 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:45:49.177322 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:45:49.177322 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:45:49.177322 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:45:49.177322 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:45:49.177322 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:45:49.177322 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 10 11:45:49.177322 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 10 11:45:49.177322 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 10 11:45:49.177322 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 10 11:45:49.178206 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 10 11:45:49.178206 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 10 11:45:49.178206 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 10 11:45:49.178206 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 10 11:45:49.178206 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:45:49.178206 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:45:49.178206 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:45:49.178206 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:45:49.178206 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:45:49.178206 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:45:49.178206 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:45:49.178206 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application with Lock( Oct 10 11:45:49.178206 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:45:49.178206 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:45:49.178206 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:45:49.178206 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:45:49.179116 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:45:49.179116 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:45:49.179116 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:45:49.179116 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:45:49.179116 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 10 11:45:49.179116 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 10 11:45:49.179116 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 10 11:45:49.179116 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 10 11:45:49.179116 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 10 11:45:49.179116 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 10 11:45:49.179116 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 10 11:45:49.179116 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Oct 10 11:45:49.179731 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 71/161] 10.222.0.46 () {56 vars in 1099 bytes} [Mon Oct 10 11:45:48 2022] GET /identity/v3/roles/member => generated 84 bytes in 420 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:45:49.188399 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40bbfb1-be47-48df-bc5b-72146cbc29d1 None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:49.266050 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:49.290975 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f2138b-b90a-4ad6-bacd-ee7c275934d0 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:49.330102 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbb4b37-d02b-48fb-af2b-19b14a5c2e2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:49.331141 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbb4b37-d02b-48fb-af2b-19b14a5c2e2d None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:49.331628 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbb4b37-d02b-48fb-af2b-19b14a5c2e2d None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:49.332798 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbb4b37-d02b-48fb-af2b-19b14a5c2e2d None admin] PATH_INFO: `/v3/projects/2d0af77a941540ef9b78b89ebae1ef1a` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:49.336630 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cbb4b37-d02b-48fb-af2b-19b14a5c2e2d None admin] RBAC: Authorizing `identity:get_project(project_id=2d0af77a941540ef9b78b89ebae1ef1a)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:49.338327 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cbb4b37-d02b-48fb-af2b-19b14a5c2e2d None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:49.344369 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 91/162] 10.222.0.46 () {56 vars in 1186 bytes} [Mon Oct 10 11:45:49 2022] GET /identity/v3/projects/2d0af77a941540ef9b78b89ebae1ef1a => generated 298 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:45:49.346558 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbb4b37-d02b-48fb-af2b-19b14a5c2e2d None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:49.450602 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40bbfb1-be47-48df-bc5b-72146cbc29d1 None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:49.468732 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:49.539005 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42168cc-4af0-4471-8c60-76a344a7ced6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:49.539729 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42168cc-4af0-4471-8c60-76a344a7ced6 None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:49.540006 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42168cc-4af0-4471-8c60-76a344a7ced6 None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:49.540280 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42168cc-4af0-4471-8c60-76a344a7ced6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:49.541301 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbb4b37-d02b-48fb-af2b-19b14a5c2e2d None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:49.563456 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42168cc-4af0-4471-8c60-76a344a7ced6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 10 11:45:49.578074 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application [None req-e42168cc-4af0-4471-8c60-76a344a7ced6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 10 11:45:49.578074 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:45:49.578074 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:45:49.578074 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:45:49.578074 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:45:49.578074 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:45:49.578074 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:45:49.578074 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:45:49.578074 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:45:49.578074 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:45:49.578074 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:45:49.578074 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:45:49.578074 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 10 11:45:49.578074 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 10 11:45:49.578074 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 10 11:45:49.578074 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 10 11:45:49.579849 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 10 11:45:49.579849 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 10 11:45:49.579849 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 10 11:45:49.579849 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 10 11:45:49.579849 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:45:49.579849 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:45:49.579849 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:45:49.579849 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:45:49.579849 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:45:49.579849 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:45:49.579849 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:45:49.579849 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application with Lock( Oct 10 11:45:49.579849 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:45:49.579849 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:45:49.579849 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:45:49.579849 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:45:49.581455 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:45:49.581455 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:45:49.581455 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:45:49.581455 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:45:49.581455 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 10 11:45:49.581455 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 10 11:45:49.581455 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 10 11:45:49.581455 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 10 11:45:49.581455 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 10 11:45:49.581455 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 10 11:45:49.581455 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 10 11:45:49.581455 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Oct 10 11:45:49.581455 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 72/163] 10.222.0.46 () {56 vars in 1096 bytes} [Mon Oct 10 11:45:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 392 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:45:49.581455 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42168cc-4af0-4471-8c60-76a344a7ced6 None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:49.608413 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ffab85-8a9a-403b-bd86-65e4612fcf98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:49.609052 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ffab85-8a9a-403b-bd86-65e4612fcf98 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:49.609340 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ffab85-8a9a-403b-bd86-65e4612fcf98 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:49.609715 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ffab85-8a9a-403b-bd86-65e4612fcf98 None admin] PATH_INFO: `/v3/roles` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:49.610287 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ffab85-8a9a-403b-bd86-65e4612fcf98 None admin] RBAC: Adding query filter params (name=member) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:45:49.610591 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ffab85-8a9a-403b-bd86-65e4612fcf98 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:49.612543 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ffab85-8a9a-403b-bd86-65e4612fcf98 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:49.627858 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 92/164] 10.222.0.46 () {56 vars in 1101 bytes} [Mon Oct 10 11:45:49 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:45:49.628723 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ffab85-8a9a-403b-bd86-65e4612fcf98 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:49.650038 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:49.756101 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42168cc-4af0-4471-8c60-76a344a7ced6 None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:49.796752 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a51632-db85-4739-8530-96498989874c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:49.797413 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a51632-db85-4739-8530-96498989874c None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:49.797670 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a51632-db85-4739-8530-96498989874c None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:49.797943 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a51632-db85-4739-8530-96498989874c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:49.799204 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a51632-db85-4739-8530-96498989874c None admin] RBAC: Adding query filter params (role.id=b2cb2a6e818e47338ce071e6fedfc673, scope.project.id=2d0af77a941540ef9b78b89ebae1ef1a, user.id=531d202e4dea4ee38988bfd8b9e26751) {{(pid=81676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:45:49.799544 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a51632-db85-4739-8530-96498989874c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:49.801676 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a51632-db85-4739-8530-96498989874c None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:49.809412 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 73/165] 10.222.0.46 () {56 vars in 1374 bytes} [Mon Oct 10 11:45:49 2022] GET /identity/v3/role_assignments?user.id=531d202e4dea4ee38988bfd8b9e26751&scope.project.id=2d0af77a941540ef9b78b89ebae1ef1a&role.id=b2cb2a6e818e47338ce071e6fedfc673 => generated 608 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:45:49.810260 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a51632-db85-4739-8530-96498989874c None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:49.874460 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:49.929131 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ffab85-8a9a-403b-bd86-65e4612fcf98 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:50.005004 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532df37f-be98-46ed-ba3c-d2eb617df0e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:50.005672 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532df37f-be98-46ed-ba3c-d2eb617df0e2 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:50.005940 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532df37f-be98-46ed-ba3c-d2eb617df0e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:50.006270 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532df37f-be98-46ed-ba3c-d2eb617df0e2 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:50.012700 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a51632-db85-4739-8530-96498989874c None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:50.034581 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532df37f-be98-46ed-ba3c-d2eb617df0e2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 10 11:45:50.045326 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application [None req-532df37f-be98-46ed-ba3c-d2eb617df0e2 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 10 11:45:50.045326 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:45:50.045326 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:45:50.045326 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:45:50.045326 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:45:50.045326 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:45:50.045326 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:45:50.045326 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:45:50.045326 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:45:50.045326 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:45:50.045326 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:45:50.045326 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:45:50.045326 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 10 11:45:50.045326 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 10 11:45:50.045326 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 10 11:45:50.045326 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 10 11:45:50.048984 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 10 11:45:50.048984 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 10 11:45:50.048984 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 10 11:45:50.048984 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 10 11:45:50.048984 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:45:50.048984 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:45:50.048984 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:45:50.048984 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:45:50.048984 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:45:50.048984 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:45:50.048984 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:45:50.048984 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application with Lock( Oct 10 11:45:50.048984 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:45:50.048984 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:45:50.048984 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:45:50.048984 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:45:50.052064 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:45:50.052064 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:45:50.052064 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:45:50.052064 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:45:50.052064 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 10 11:45:50.052064 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 10 11:45:50.052064 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 10 11:45:50.052064 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 10 11:45:50.052064 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 10 11:45:50.052064 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 10 11:45:50.052064 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 10 11:45:50.052064 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Oct 10 11:45:50.052064 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 93/166] 10.222.0.46 () {56 vars in 1114 bytes} [Mon Oct 10 11:45:49 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 418 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:45:50.052064 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532df37f-be98-46ed-ba3c-d2eb617df0e2 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:50.086863 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1add1a90-d603-4103-9edc-0e9c50b870ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:50.087483 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1add1a90-d603-4103-9edc-0e9c50b870ef None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:50.088360 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1add1a90-d603-4103-9edc-0e9c50b870ef None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:50.088618 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1add1a90-d603-4103-9edc-0e9c50b870ef None admin] PATH_INFO: `/v3/roles` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:50.089471 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1add1a90-d603-4103-9edc-0e9c50b870ef None admin] RBAC: Adding query filter params (name=admin) {{(pid=81676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:45:50.089780 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1add1a90-d603-4103-9edc-0e9c50b870ef None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:50.091301 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1add1a90-d603-4103-9edc-0e9c50b870ef None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:50.101780 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 74/167] 10.222.0.46 () {56 vars in 1099 bytes} [Mon Oct 10 11:45:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:45:50.102658 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1add1a90-d603-4103-9edc-0e9c50b870ef None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:50.256741 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:50.263638 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:50.271669 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1add1a90-d603-4103-9edc-0e9c50b870ef None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:50.342696 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532df37f-be98-46ed-ba3c-d2eb617df0e2 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:50.385372 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae416ae2-34d9-441a-a37b-f7c91eb2e468 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:50.386427 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae416ae2-34d9-441a-a37b-f7c91eb2e468 None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:50.386956 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae416ae2-34d9-441a-a37b-f7c91eb2e468 None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:50.387500 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae416ae2-34d9-441a-a37b-f7c91eb2e468 None admin] PATH_INFO: `/v3/roles` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:50.389014 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae416ae2-34d9-441a-a37b-f7c91eb2e468 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:45:50.389573 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae416ae2-34d9-441a-a37b-f7c91eb2e468 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:50.392122 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae416ae2-34d9-441a-a37b-f7c91eb2e468 None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:50.420495 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 75/168] 10.222.0.46 () {56 vars in 1111 bytes} [Mon Oct 10 11:45:50 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 318 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:45:50.421968 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae416ae2-34d9-441a-a37b-f7c91eb2e468 None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:50.435401 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df22aa37-838d-4b09-9490-3e973d7552dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:50.436082 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df22aa37-838d-4b09-9490-3e973d7552dd None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:50.436823 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df22aa37-838d-4b09-9490-3e973d7552dd None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:50.436823 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df22aa37-838d-4b09-9490-3e973d7552dd None admin] PATH_INFO: `/v3/users/531d202e4dea4ee38988bfd8b9e26751` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:50.437812 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df22aa37-838d-4b09-9490-3e973d7552dd None admin] RBAC: Authorizing `identity:get_user(user_id=531d202e4dea4ee38988bfd8b9e26751)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:50.439412 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df22aa37-838d-4b09-9490-3e973d7552dd None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:50.441040 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 94/169] 10.222.0.46 () {56 vars in 1177 bytes} [Mon Oct 10 11:45:50 2022] GET /identity/v3/users/531d202e4dea4ee38988bfd8b9e26751 => generated 275 bytes in 394 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:45:50.441830 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df22aa37-838d-4b09-9490-3e973d7552dd None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:50.683311 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df22aa37-838d-4b09-9490-3e973d7552dd None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:50.723388 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae416ae2-34d9-441a-a37b-f7c91eb2e468 None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:50.726294 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babdf541-9c08-43af-a89e-cb77d1114128 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:50.727819 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babdf541-9c08-43af-a89e-cb77d1114128 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:50.727953 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babdf541-9c08-43af-a89e-cb77d1114128 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:50.728152 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babdf541-9c08-43af-a89e-cb77d1114128 None admin] PATH_INFO: `/v3/users/531d202e4dea4ee38988bfd8b9e26751` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:50.737005 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-babdf541-9c08-43af-a89e-cb77d1114128 None admin] RBAC: Authorizing `identity:get_user(user_id=531d202e4dea4ee38988bfd8b9e26751)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:50.737005 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-babdf541-9c08-43af-a89e-cb77d1114128 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:50.737005 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 95/170] 10.222.0.46 () {56 vars in 1177 bytes} [Mon Oct 10 11:45:50 2022] GET /identity/v3/users/531d202e4dea4ee38988bfd8b9e26751 => generated 275 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:45:50.737769 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babdf541-9c08-43af-a89e-cb77d1114128 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:50.741740 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:50.787886 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f053f876-81da-4f46-b0e7-dc8a9ead5625 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:50.788935 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f053f876-81da-4f46-b0e7-dc8a9ead5625 None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:50.789431 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f053f876-81da-4f46-b0e7-dc8a9ead5625 None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:50.789915 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f053f876-81da-4f46-b0e7-dc8a9ead5625 None admin] PATH_INFO: `/v3/users/8cbbc9233ced43d487079e9479dc555c` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:50.791421 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f053f876-81da-4f46-b0e7-dc8a9ead5625 None admin] RBAC: Authorizing `identity:get_user(user_id=8cbbc9233ced43d487079e9479dc555c)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:50.793204 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f053f876-81da-4f46-b0e7-dc8a9ead5625 None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:50.795162 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 76/171] 10.222.0.46 () {56 vars in 1177 bytes} [Mon Oct 10 11:45:50 2022] GET /identity/v3/users/8cbbc9233ced43d487079e9479dc555c => generated 247 bytes in 374 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:45:50.796243 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f053f876-81da-4f46-b0e7-dc8a9ead5625 None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:50.830090 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:50.856464 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babdf541-9c08-43af-a89e-cb77d1114128 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:50.935102 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ca6f12-56ee-4341-bade-91217656c51d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:50.936659 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ca6f12-56ee-4341-bade-91217656c51d None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:50.937178 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ca6f12-56ee-4341-bade-91217656c51d None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:50.937678 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ca6f12-56ee-4341-bade-91217656c51d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:51.014710 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f053f876-81da-4f46-b0e7-dc8a9ead5625 None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:51.104662 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ca6f12-56ee-4341-bade-91217656c51d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:51.107770 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ca6f12-56ee-4341-bade-91217656c51d None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:51.108899 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8757e05e-0d4e-4237-8686-a441e66dbd5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:51.109531 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8757e05e-0d4e-4237-8686-a441e66dbd5b None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:51.109814 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8757e05e-0d4e-4237-8686-a441e66dbd5b None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:51.110096 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8757e05e-0d4e-4237-8686-a441e66dbd5b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:51.133934 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:51.290607 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8757e05e-0d4e-4237-8686-a441e66dbd5b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:51.292099 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8757e05e-0d4e-4237-8686-a441e66dbd5b None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:51.327546 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 96/172] 10.222.0.46 () {58 vars in 1282 bytes} [Mon Oct 10 11:45:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 590 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 10 11:45:51.329351 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ca6f12-56ee-4341-bade-91217656c51d None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:51.364019 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:51.460539 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ca6f12-56ee-4341-bade-91217656c51d None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:51.463438 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:51.503375 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae9bd12-e341-4635-a4a1-c681696a95f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:51.504474 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae9bd12-e341-4635-a4a1-c681696a95f3 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:51.504994 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae9bd12-e341-4635-a4a1-c681696a95f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:51.505473 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae9bd12-e341-4635-a4a1-c681696a95f3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:51.569849 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 77/173] 10.222.0.46 () {58 vars in 1282 bytes} [Mon Oct 10 11:45:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 774 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 10 11:45:51.570649 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8757e05e-0d4e-4237-8686-a441e66dbd5b None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:51.584816 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae9bd12-e341-4635-a4a1-c681696a95f3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:51.586718 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae9bd12-e341-4635-a4a1-c681696a95f3 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:51.698954 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 97/174] 10.222.0.46 () {58 vars in 1282 bytes} [Mon Oct 10 11:45:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 10 11:45:51.700234 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae9bd12-e341-4635-a4a1-c681696a95f3 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:51.703816 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:51.800910 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8757e05e-0d4e-4237-8686-a441e66dbd5b None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:51.813218 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae9bd12-e341-4635-a4a1-c681696a95f3 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:51.828108 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:51.841015 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80e7580-f2eb-46bf-a08f-c8457bde1cff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:51.842348 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80e7580-f2eb-46bf-a08f-c8457bde1cff None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:51.842601 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80e7580-f2eb-46bf-a08f-c8457bde1cff None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:51.843225 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80e7580-f2eb-46bf-a08f-c8457bde1cff None admin] PATH_INFO: `/v3/projects/7c32e0f418e949da8cd9bbe662b13191` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:51.844777 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80e7580-f2eb-46bf-a08f-c8457bde1cff None admin] RBAC: Authorizing `identity:get_project(project_id=7c32e0f418e949da8cd9bbe662b13191)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:51.847006 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80e7580-f2eb-46bf-a08f-c8457bde1cff None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:51.849041 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 78/175] 10.222.0.46 () {56 vars in 1186 bytes} [Mon Oct 10 11:45:51 2022] GET /identity/v3/projects/7c32e0f418e949da8cd9bbe662b13191 => generated 312 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:45:51.854126 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80e7580-f2eb-46bf-a08f-c8457bde1cff None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:51.855993 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbae16ff-e8ea-4d7e-adf2-8afbf3579e8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:51.855993 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbae16ff-e8ea-4d7e-adf2-8afbf3579e8d None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:51.856297 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbae16ff-e8ea-4d7e-adf2-8afbf3579e8d None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:51.856573 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbae16ff-e8ea-4d7e-adf2-8afbf3579e8d None admin] PATH_INFO: `/v3/projects/2d0af77a941540ef9b78b89ebae1ef1a` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:51.857795 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbae16ff-e8ea-4d7e-adf2-8afbf3579e8d None admin] RBAC: Authorizing `identity:get_project(project_id=2d0af77a941540ef9b78b89ebae1ef1a)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:51.859535 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbae16ff-e8ea-4d7e-adf2-8afbf3579e8d None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:51.868335 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 98/176] 10.222.0.46 () {56 vars in 1186 bytes} [Mon Oct 10 11:45:51 2022] GET /identity/v3/projects/2d0af77a941540ef9b78b89ebae1ef1a => generated 298 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:45:51.869931 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065b1c0c-a7eb-4a8f-b8be-31f7c77886a1 None None] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:51.870216 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065b1c0c-a7eb-4a8f-b8be-31f7c77886a1 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:51.870495 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065b1c0c-a7eb-4a8f-b8be-31f7c77886a1 None None] PATH_INFO: `/` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:51.874972 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 99/177] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 10 11:45:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:45:51.876500 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc5d420-e53f-4eee-835d-9f19c9297316 None None] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:51.877574 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc5d420-e53f-4eee-835d-9f19c9297316 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:51.878440 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc5d420-e53f-4eee-835d-9f19c9297316 None None] PATH_INFO: `/` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:51.880130 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 100/178] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 10 11:45:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:45:51.880878 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc5d420-e53f-4eee-835d-9f19c9297316 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:51.976354 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:51.993587 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80e7580-f2eb-46bf-a08f-c8457bde1cff None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:52.013104 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc5d420-e53f-4eee-835d-9f19c9297316 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:52.039506 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4717525-b883-4ea7-a9cf-96dff8c45841 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:52.040284 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4717525-b883-4ea7-a9cf-96dff8c45841 None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:52.040571 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4717525-b883-4ea7-a9cf-96dff8c45841 None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:52.040849 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4717525-b883-4ea7-a9cf-96dff8c45841 None admin] PATH_INFO: `/v3/projects/2d0af77a941540ef9b78b89ebae1ef1a` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:52.042645 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4717525-b883-4ea7-a9cf-96dff8c45841 None admin] RBAC: Authorizing `identity:get_project(project_id=2d0af77a941540ef9b78b89ebae1ef1a)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:52.044274 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4717525-b883-4ea7-a9cf-96dff8c45841 None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:52.050738 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 79/179] 10.222.0.46 () {56 vars in 1186 bytes} [Mon Oct 10 11:45:51 2022] GET /identity/v3/projects/2d0af77a941540ef9b78b89ebae1ef1a => generated 298 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:45:52.051850 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4717525-b883-4ea7-a9cf-96dff8c45841 None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:52.058095 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e0d06c-c0b5-4f4c-acef-c3c8d6ed1e5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:52.059081 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e0d06c-c0b5-4f4c-acef-c3c8d6ed1e5d None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:52.059580 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e0d06c-c0b5-4f4c-acef-c3c8d6ed1e5d None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:52.060041 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e0d06c-c0b5-4f4c-acef-c3c8d6ed1e5d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:52.060960 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e0d06c-c0b5-4f4c-acef-c3c8d6ed1e5d None admin] RBAC: Adding query filter params (role.id=b2cb2a6e818e47338ce071e6fedfc673, scope.project.id=7c32e0f418e949da8cd9bbe662b13191, user.id=531d202e4dea4ee38988bfd8b9e26751) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:45:52.061404 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e0d06c-c0b5-4f4c-acef-c3c8d6ed1e5d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:52.063233 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e0d06c-c0b5-4f4c-acef-c3c8d6ed1e5d None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:52.070899 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 101/180] 10.222.0.46 () {56 vars in 1374 bytes} [Mon Oct 10 11:45:51 2022] GET /identity/v3/role_assignments?user.id=531d202e4dea4ee38988bfd8b9e26751&scope.project.id=7c32e0f418e949da8cd9bbe662b13191&role.id=b2cb2a6e818e47338ce071e6fedfc673 => generated 608 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:45:52.080838 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e172918c-d0e0-4e3d-ba8b-10a52cf2f97b None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:52.081048 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e172918c-d0e0-4e3d-ba8b-10a52cf2f97b None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:52.081325 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e172918c-d0e0-4e3d-ba8b-10a52cf2f97b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:52.102247 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-e172918c-d0e0-4e3d-ba8b-10a52cf2f97b None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:45:52.103525 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-e172918c-d0e0-4e3d-ba8b-10a52cf2f97b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:45:52.139062 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:52.175542 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4717525-b883-4ea7-a9cf-96dff8c45841 None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:52.186847 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 102/181] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 10 11:45:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 10 11:45:52.188467 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2f3c75-96f5-4ca0-a0fa-c3485dd90f67 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:52.188754 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2f3c75-96f5-4ca0-a0fa-c3485dd90f67 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:52.189028 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2f3c75-96f5-4ca0-a0fa-c3485dd90f67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:52.228821 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-8c2f3c75-96f5-4ca0-a0fa-c3485dd90f67 None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:45:52.230393 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-8c2f3c75-96f5-4ca0-a0fa-c3485dd90f67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:45:52.255103 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314a4bce-9146-455e-b7d4-a071351acdb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:52.256465 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314a4bce-9146-455e-b7d4-a071351acdb7 None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:52.259218 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314a4bce-9146-455e-b7d4-a071351acdb7 None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:52.259796 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314a4bce-9146-455e-b7d4-a071351acdb7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:52.260804 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314a4bce-9146-455e-b7d4-a071351acdb7 None admin] RBAC: Adding query filter params (role.id=6762beebc2bf4e6f804cbbe55a475e70, scope.project.id=2d0af77a941540ef9b78b89ebae1ef1a, user.id=531d202e4dea4ee38988bfd8b9e26751) {{(pid=81676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:45:52.261652 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314a4bce-9146-455e-b7d4-a071351acdb7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:52.265005 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314a4bce-9146-455e-b7d4-a071351acdb7 None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:52.279332 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 80/182] 10.222.0.46 () {56 vars in 1374 bytes} [Mon Oct 10 11:45:52 2022] GET /identity/v3/role_assignments?user.id=531d202e4dea4ee38988bfd8b9e26751&scope.project.id=2d0af77a941540ef9b78b89ebae1ef1a&role.id=6762beebc2bf4e6f804cbbe55a475e70 => generated 608 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:45:52.279332 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314a4bce-9146-455e-b7d4-a071351acdb7 None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:52.367125 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:52.377896 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 103/183] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 10 11:45:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 10 11:45:52.381717 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9640d18b-4e17-42cc-8f5b-ffea8e41a532 None None] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:52.381717 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9640d18b-4e17-42cc-8f5b-ffea8e41a532 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:52.381717 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9640d18b-4e17-42cc-8f5b-ffea8e41a532 None None] PATH_INFO: `/` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:52.384308 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 104/184] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 10 11:45:52 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:45:52.385656 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21437909-991b-4bc1-a645-186beff35229 None None] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:52.385946 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21437909-991b-4bc1-a645-186beff35229 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:52.386226 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21437909-991b-4bc1-a645-186beff35229 None None] PATH_INFO: `/` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:52.387827 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 105/185] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 10 11:45:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:45:52.394348 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf012ea6-d33d-48fc-b1e0-5b6ac68ca7f0 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:52.394629 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf012ea6-d33d-48fc-b1e0-5b6ac68ca7f0 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:52.394943 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf012ea6-d33d-48fc-b1e0-5b6ac68ca7f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:52.414913 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-bf012ea6-d33d-48fc-b1e0-5b6ac68ca7f0 None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:45:52.416134 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-bf012ea6-d33d-48fc-b1e0-5b6ac68ca7f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:45:52.518481 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314a4bce-9146-455e-b7d4-a071351acdb7 None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:52.553279 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 106/186] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 10 11:45:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 10 11:45:52.559038 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0adfe23-28c8-4076-bed1-801fab6c8b09 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:52.559361 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0adfe23-28c8-4076-bed1-801fab6c8b09 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:52.559642 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0adfe23-28c8-4076-bed1-801fab6c8b09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:52.568084 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:52.580922 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ee46da-df1e-48ce-866e-3a3cab48d699 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:52.581644 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ee46da-df1e-48ce-866e-3a3cab48d699 None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:52.581923 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ee46da-df1e-48ce-866e-3a3cab48d699 None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:52.582248 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ee46da-df1e-48ce-866e-3a3cab48d699 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:52.584026 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ee46da-df1e-48ce-866e-3a3cab48d699 None admin] RBAC: Adding query filter params (role.id=0d822a0674b6495fac623877a3b38025, scope.project.id=2d0af77a941540ef9b78b89ebae1ef1a, user.id=8cbbc9233ced43d487079e9479dc555c) {{(pid=81676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:45:52.584345 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ee46da-df1e-48ce-866e-3a3cab48d699 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:52.585237 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-f0adfe23-28c8-4076-bed1-801fab6c8b09 None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:45:52.585818 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ee46da-df1e-48ce-866e-3a3cab48d699 None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:52.586430 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-f0adfe23-28c8-4076-bed1-801fab6c8b09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:45:52.592846 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 81/187] 10.222.0.46 () {56 vars in 1374 bytes} [Mon Oct 10 11:45:52 2022] GET /identity/v3/role_assignments?user.id=8cbbc9233ced43d487079e9479dc555c&scope.project.id=2d0af77a941540ef9b78b89ebae1ef1a&role.id=0d822a0674b6495fac623877a3b38025 => generated 608 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:45:52.593753 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ee46da-df1e-48ce-866e-3a3cab48d699 None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:52.724342 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 107/188] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 10 11:45:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 10 11:45:52.757248 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0adfe23-28c8-4076-bed1-801fab6c8b09 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:52.773508 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ee46da-df1e-48ce-866e-3a3cab48d699 None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:52.852643 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69516e0f-8f8a-47ad-81b7-bcd196288e1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:52.853406 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69516e0f-8f8a-47ad-81b7-bcd196288e1a None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:52.853597 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69516e0f-8f8a-47ad-81b7-bcd196288e1a None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:52.853844 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69516e0f-8f8a-47ad-81b7-bcd196288e1a None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:52.870366 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:52.883271 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69516e0f-8f8a-47ad-81b7-bcd196288e1a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 10 11:45:52.890702 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application [None req-69516e0f-8f8a-47ad-81b7-bcd196288e1a None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 10 11:45:52.890702 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:45:52.890702 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:45:52.890702 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:45:52.890702 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:45:52.890702 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:45:52.890702 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:45:52.890702 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:45:52.890702 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:45:52.890702 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:45:52.890702 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:45:52.890702 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:45:52.890702 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 10 11:45:52.890702 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 10 11:45:52.890702 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 10 11:45:52.890702 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 10 11:45:52.891617 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 10 11:45:52.891617 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 10 11:45:52.891617 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 10 11:45:52.891617 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 10 11:45:52.891617 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:45:52.891617 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:45:52.891617 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:45:52.891617 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:45:52.891617 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:45:52.891617 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:45:52.891617 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:45:52.891617 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application with Lock( Oct 10 11:45:52.891617 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:45:52.891617 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:45:52.891617 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:45:52.891617 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:45:52.892430 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:45:52.892430 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:45:52.892430 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:45:52.892430 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:45:52.892430 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 10 11:45:52.892430 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 10 11:45:52.892430 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 10 11:45:52.892430 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 10 11:45:52.892430 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 10 11:45:52.892430 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 10 11:45:52.892430 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 10 11:45:52.892430 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Oct 10 11:45:52.895987 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 82/189] 10.222.0.46 () {56 vars in 1099 bytes} [Mon Oct 10 11:45:52 2022] GET /identity/v3/roles/reader => generated 84 bytes in 303 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:45:52.897125 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69516e0f-8f8a-47ad-81b7-bcd196288e1a None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:52.945546 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0adfe23-28c8-4076-bed1-801fab6c8b09 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:52.982922 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:53.001078 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfd6e8a-77f6-464d-86d6-8c31110de693 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:53.002631 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfd6e8a-77f6-464d-86d6-8c31110de693 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:53.003140 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfd6e8a-77f6-464d-86d6-8c31110de693 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:53.003586 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfd6e8a-77f6-464d-86d6-8c31110de693 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:53.024762 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bfd6e8a-77f6-464d-86d6-8c31110de693 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 10 11:45:53.036477 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application [None req-6bfd6e8a-77f6-464d-86d6-8c31110de693 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 10 11:45:53.036477 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:45:53.036477 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:45:53.036477 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:45:53.036477 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:45:53.036477 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:45:53.036477 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:45:53.036477 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:45:53.036477 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:45:53.036477 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:45:53.036477 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:45:53.036477 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:45:53.036477 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 10 11:45:53.036477 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 10 11:45:53.036477 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 10 11:45:53.036477 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 10 11:45:53.037567 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 10 11:45:53.037567 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 10 11:45:53.037567 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 10 11:45:53.037567 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 10 11:45:53.037567 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:45:53.037567 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:45:53.037567 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:45:53.037567 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:45:53.037567 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:45:53.037567 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:45:53.037567 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:45:53.037567 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application with Lock( Oct 10 11:45:53.037567 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:45:53.037567 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:45:53.037567 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:45:53.037567 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:45:53.038492 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:45:53.038492 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:45:53.038492 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:45:53.038492 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:45:53.038492 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 10 11:45:53.038492 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 10 11:45:53.038492 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 10 11:45:53.038492 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 10 11:45:53.038492 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 10 11:45:53.038492 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 10 11:45:53.038492 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 10 11:45:53.038492 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Oct 10 11:45:53.038492 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 108/190] 10.222.0.46 () {56 vars in 1096 bytes} [Mon Oct 10 11:45:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 10 11:45:53.039274 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfd6e8a-77f6-464d-86d6-8c31110de693 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:53.054755 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69516e0f-8f8a-47ad-81b7-bcd196288e1a None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:53.089680 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b637a5b-272d-418d-88b6-e081e0406353 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:53.090315 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b637a5b-272d-418d-88b6-e081e0406353 None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:53.090595 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b637a5b-272d-418d-88b6-e081e0406353 None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:53.090893 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b637a5b-272d-418d-88b6-e081e0406353 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:53.092010 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b637a5b-272d-418d-88b6-e081e0406353 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:53.093462 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b637a5b-272d-418d-88b6-e081e0406353 None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:53.095081 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 83/191] 10.222.0.46 () {56 vars in 1108 bytes} [Mon Oct 10 11:45:52 2022] GET /identity/v3/domains/default => generated 199 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:45:53.096383 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b637a5b-272d-418d-88b6-e081e0406353 None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:53.144992 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfd6e8a-77f6-464d-86d6-8c31110de693 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:53.162127 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:53.180914 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d090940-c929-467d-8027-bd3240f2ac56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:53.181529 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d090940-c929-467d-8027-bd3240f2ac56 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:53.181806 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d090940-c929-467d-8027-bd3240f2ac56 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:53.182127 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d090940-c929-467d-8027-bd3240f2ac56 None admin] PATH_INFO: `/v3/roles` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:53.182836 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d090940-c929-467d-8027-bd3240f2ac56 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:45:53.183353 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d090940-c929-467d-8027-bd3240f2ac56 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:53.185208 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d090940-c929-467d-8027-bd3240f2ac56 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:53.193690 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 109/192] 10.222.0.46 () {56 vars in 1101 bytes} [Mon Oct 10 11:45:53 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:45:53.194513 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d090940-c929-467d-8027-bd3240f2ac56 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:53.202080 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b637a5b-272d-418d-88b6-e081e0406353 None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:53.219007 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:53.237121 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2871497c-eb95-43c2-8424-3a416ce0f512 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:53.237674 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2871497c-eb95-43c2-8424-3a416ce0f512 None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:53.237955 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2871497c-eb95-43c2-8424-3a416ce0f512 None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:53.238231 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2871497c-eb95-43c2-8424-3a416ce0f512 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:53.249268 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2871497c-eb95-43c2-8424-3a416ce0f512 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 10 11:45:53.254658 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application [None req-2871497c-eb95-43c2-8424-3a416ce0f512 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 10 11:45:53.254658 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:45:53.254658 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:45:53.254658 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:45:53.254658 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:45:53.254658 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:45:53.254658 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:45:53.254658 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:45:53.254658 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:45:53.254658 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:45:53.254658 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:45:53.254658 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:45:53.254658 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 10 11:45:53.254658 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 10 11:45:53.254658 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 10 11:45:53.254658 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 10 11:45:53.255682 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 10 11:45:53.255682 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 10 11:45:53.255682 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 10 11:45:53.255682 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 10 11:45:53.255682 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:45:53.255682 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:45:53.255682 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:45:53.255682 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:45:53.255682 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:45:53.255682 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:45:53.255682 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:45:53.255682 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application with Lock( Oct 10 11:45:53.255682 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:45:53.255682 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:45:53.255682 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:45:53.255682 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:45:53.259443 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:45:53.259443 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:45:53.259443 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:45:53.259443 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:45:53.259443 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 10 11:45:53.259443 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 10 11:45:53.259443 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 10 11:45:53.259443 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 10 11:45:53.259443 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 10 11:45:53.259443 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 10 11:45:53.259443 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 10 11:45:53.259443 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Oct 10 11:45:53.259443 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 84/193] 10.222.0.46 () {56 vars in 1114 bytes} [Mon Oct 10 11:45:53 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:45:53.259443 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2871497c-eb95-43c2-8424-3a416ce0f512 None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:53.306290 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d090940-c929-467d-8027-bd3240f2ac56 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:53.346032 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f837e192-8740-42a7-bf43-5a9a3601dc94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:53.346714 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f837e192-8740-42a7-bf43-5a9a3601dc94 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:53.347117 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f837e192-8740-42a7-bf43-5a9a3601dc94 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:53.347397 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f837e192-8740-42a7-bf43-5a9a3601dc94 None admin] PATH_INFO: `/v3/roles` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:53.348069 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f837e192-8740-42a7-bf43-5a9a3601dc94 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:45:53.348370 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f837e192-8740-42a7-bf43-5a9a3601dc94 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:53.349749 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f837e192-8740-42a7-bf43-5a9a3601dc94 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:53.358433 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 110/194] 10.222.0.46 () {56 vars in 1099 bytes} [Mon Oct 10 11:45:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:45:53.359389 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f837e192-8740-42a7-bf43-5a9a3601dc94 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:53.374475 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2871497c-eb95-43c2-8424-3a416ce0f512 None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:53.403391 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:53.410744 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b54e73-07e4-469d-b755-432d27f6ba3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:53.414409 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b54e73-07e4-469d-b755-432d27f6ba3c None admin] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:53.414409 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b54e73-07e4-469d-b755-432d27f6ba3c None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:53.414409 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b54e73-07e4-469d-b755-432d27f6ba3c None admin] PATH_INFO: `/v3/users` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:53.414409 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b54e73-07e4-469d-b755-432d27f6ba3c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:53.414409 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b54e73-07e4-469d-b755-432d27f6ba3c None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:53.463777 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f837e192-8740-42a7-bf43-5a9a3601dc94 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:53.498339 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c8bd85-244e-476d-a1f3-4a37448b2ca4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:53.498952 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c8bd85-244e-476d-a1f3-4a37448b2ca4 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:53.499235 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c8bd85-244e-476d-a1f3-4a37448b2ca4 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:53.499511 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c8bd85-244e-476d-a1f3-4a37448b2ca4 None admin] PATH_INFO: `/v3/users/ae40cb146dee491b8706cc0d8a34df65` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:53.500637 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c8bd85-244e-476d-a1f3-4a37448b2ca4 None admin] RBAC: Authorizing `identity:get_user(user_id=ae40cb146dee491b8706cc0d8a34df65)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:53.502155 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c8bd85-244e-476d-a1f3-4a37448b2ca4 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:53.503768 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 111/195] 10.222.0.46 () {56 vars in 1177 bytes} [Mon Oct 10 11:45:53 2022] GET /identity/v3/users/ae40cb146dee491b8706cc0d8a34df65 => generated 289 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:45:53.504538 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c8bd85-244e-476d-a1f3-4a37448b2ca4 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:53.531544 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 85/196] 10.222.0.46 () {60 vars in 1132 bytes} [Mon Oct 10 11:45:53 2022] POST /identity/v3/users => generated 297 bytes in 275 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 11:45:53.532387 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b54e73-07e4-469d-b755-432d27f6ba3c None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:53.574425 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:53.625980 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c8bd85-244e-476d-a1f3-4a37448b2ca4 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:53.638442 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:53.662379 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b54e73-07e4-469d-b755-432d27f6ba3c None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:53.670915 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ffcab9-3e0a-43fd-b866-e762822b791f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:53.671552 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ffcab9-3e0a-43fd-b866-e762822b791f None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:53.671835 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ffcab9-3e0a-43fd-b866-e762822b791f None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:53.672795 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ffcab9-3e0a-43fd-b866-e762822b791f None admin] PATH_INFO: `/v3/roles` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:53.673582 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ffcab9-3e0a-43fd-b866-e762822b791f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:45:53.674399 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ffcab9-3e0a-43fd-b866-e762822b791f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:53.676298 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ffcab9-3e0a-43fd-b866-e762822b791f None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:53.688115 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 112/197] 10.222.0.46 () {56 vars in 1111 bytes} [Mon Oct 10 11:45:53 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:45:53.688944 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ffcab9-3e0a-43fd-b866-e762822b791f None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:53.712675 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5242ada6-1ef7-49cf-a43d-94bf377a0660 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:53.713317 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5242ada6-1ef7-49cf-a43d-94bf377a0660 None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:53.713593 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5242ada6-1ef7-49cf-a43d-94bf377a0660 None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:53.713861 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5242ada6-1ef7-49cf-a43d-94bf377a0660 None admin] PATH_INFO: `/v3/users/6c3317696b14428a8edb1085c6cfcc3d` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:53.733161 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5242ada6-1ef7-49cf-a43d-94bf377a0660 None admin] RBAC: Authorizing `identity:get_user(user_id=6c3317696b14428a8edb1085c6cfcc3d)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:53.734372 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5242ada6-1ef7-49cf-a43d-94bf377a0660 None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:53.736789 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 86/198] 10.222.0.46 () {56 vars in 1177 bytes} [Mon Oct 10 11:45:53 2022] GET /identity/v3/users/6c3317696b14428a8edb1085c6cfcc3d => generated 283 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:45:53.737494 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5242ada6-1ef7-49cf-a43d-94bf377a0660 None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:53.814721 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ffcab9-3e0a-43fd-b866-e762822b791f None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:53.863959 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a33de0-d45d-438b-8d65-572c2ebd7235 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:53.863959 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a33de0-d45d-438b-8d65-572c2ebd7235 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:53.863959 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a33de0-d45d-438b-8d65-572c2ebd7235 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:53.864448 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a33de0-d45d-438b-8d65-572c2ebd7235 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:53.872896 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:53.887791 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5242ada6-1ef7-49cf-a43d-94bf377a0660 None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:53.925438 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:53.935355 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d62d55-23ed-476b-9e3d-85cc638685d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:53.939561 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d62d55-23ed-476b-9e3d-85cc638685d6 None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:53.939561 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d62d55-23ed-476b-9e3d-85cc638685d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:53.939561 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d62d55-23ed-476b-9e3d-85cc638685d6 None admin] PATH_INFO: `/v3/users/6c3317696b14428a8edb1085c6cfcc3d` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:53.939561 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d62d55-23ed-476b-9e3d-85cc638685d6 None admin] RBAC: Authorizing `identity:get_user(user_id=6c3317696b14428a8edb1085c6cfcc3d)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:53.939561 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d62d55-23ed-476b-9e3d-85cc638685d6 None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:53.940841 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 87/199] 10.222.0.46 () {56 vars in 1177 bytes} [Mon Oct 10 11:45:53 2022] GET /identity/v3/users/6c3317696b14428a8edb1085c6cfcc3d => generated 283 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:45:53.941618 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d62d55-23ed-476b-9e3d-85cc638685d6 None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:53.970135 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a33de0-d45d-438b-8d65-572c2ebd7235 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:53.972268 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a33de0-d45d-438b-8d65-572c2ebd7235 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:54.060888 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d62d55-23ed-476b-9e3d-85cc638685d6 None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:54.098370 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f67be81-2b61-47fd-867c-8fd8e3ad1661 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:54.098977 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f67be81-2b61-47fd-867c-8fd8e3ad1661 None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:54.099279 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f67be81-2b61-47fd-867c-8fd8e3ad1661 None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:54.099555 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f67be81-2b61-47fd-867c-8fd8e3ad1661 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:54.130552 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:54.143734 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 113/200] 10.222.0.46 () {58 vars in 1282 bytes} [Mon Oct 10 11:45:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 455 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 10 11:45:54.145650 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a33de0-d45d-438b-8d65-572c2ebd7235 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:54.180597 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f67be81-2b61-47fd-867c-8fd8e3ad1661 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:54.183005 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f67be81-2b61-47fd-867c-8fd8e3ad1661 None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:54.226686 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:54.258847 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a33de0-d45d-438b-8d65-572c2ebd7235 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:54.298566 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e09b19-3749-45bb-8ec8-50778fac5624 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:54.299218 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e09b19-3749-45bb-8ec8-50778fac5624 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:54.299502 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e09b19-3749-45bb-8ec8-50778fac5624 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:54.299778 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e09b19-3749-45bb-8ec8-50778fac5624 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:54.319089 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 88/201] 10.222.0.46 () {58 vars in 1282 bytes} [Mon Oct 10 11:45:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 10 11:45:54.321101 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f67be81-2b61-47fd-867c-8fd8e3ad1661 None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:54.380460 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e09b19-3749-45bb-8ec8-50778fac5624 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:54.382177 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e09b19-3749-45bb-8ec8-50778fac5624 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:54.401445 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:54.466661 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:54.468984 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f67be81-2b61-47fd-867c-8fd8e3ad1661 None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:54.503702 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 114/202] 10.222.0.46 () {58 vars in 1282 bytes} [Mon Oct 10 11:45:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 10 11:45:54.505367 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e09b19-3749-45bb-8ec8-50778fac5624 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:54.515452 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1834a705-3212-4d70-b9c5-90ef7997a126 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:54.516371 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1834a705-3212-4d70-b9c5-90ef7997a126 None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:54.516789 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1834a705-3212-4d70-b9c5-90ef7997a126 None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:54.518267 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1834a705-3212-4d70-b9c5-90ef7997a126 None admin] PATH_INFO: `/v3/projects/2d0af77a941540ef9b78b89ebae1ef1a` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:54.520079 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1834a705-3212-4d70-b9c5-90ef7997a126 None admin] RBAC: Authorizing `identity:get_project(project_id=2d0af77a941540ef9b78b89ebae1ef1a)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:54.522275 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1834a705-3212-4d70-b9c5-90ef7997a126 None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:54.525356 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 89/203] 10.222.0.46 () {56 vars in 1186 bytes} [Mon Oct 10 11:45:54 2022] GET /identity/v3/projects/2d0af77a941540ef9b78b89ebae1ef1a => generated 298 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:45:54.527108 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1834a705-3212-4d70-b9c5-90ef7997a126 None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:54.630585 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e09b19-3749-45bb-8ec8-50778fac5624 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:54.646238 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1834a705-3212-4d70-b9c5-90ef7997a126 None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:54.673492 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:54.677264 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1148de2-1da8-4d98-b402-844877e43fc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:54.678066 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1148de2-1da8-4d98-b402-844877e43fc9 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:54.678472 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1148de2-1da8-4d98-b402-844877e43fc9 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:54.678915 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1148de2-1da8-4d98-b402-844877e43fc9 None admin] PATH_INFO: `/v3/projects/55927449396d4c4f8470544dd7a23ed9` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:54.680280 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1148de2-1da8-4d98-b402-844877e43fc9 None admin] RBAC: Authorizing `identity:get_project(project_id=55927449396d4c4f8470544dd7a23ed9)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:54.681951 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1148de2-1da8-4d98-b402-844877e43fc9 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:54.683643 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3598d9-d60d-414a-9285-71404129c3b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:54.685843 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 115/204] 10.222.0.46 () {56 vars in 1186 bytes} [Mon Oct 10 11:45:54 2022] GET /identity/v3/projects/55927449396d4c4f8470544dd7a23ed9 => generated 302 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:45:54.685984 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3598d9-d60d-414a-9285-71404129c3b0 None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:54.685984 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3598d9-d60d-414a-9285-71404129c3b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:54.685984 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3598d9-d60d-414a-9285-71404129c3b0 None admin] PATH_INFO: `/v3/projects/55927449396d4c4f8470544dd7a23ed9` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:54.686211 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1148de2-1da8-4d98-b402-844877e43fc9 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:54.686708 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3598d9-d60d-414a-9285-71404129c3b0 None admin] RBAC: Authorizing `identity:get_project(project_id=55927449396d4c4f8470544dd7a23ed9)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:54.688369 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3598d9-d60d-414a-9285-71404129c3b0 None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:54.690640 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 90/205] 10.222.0.46 () {56 vars in 1186 bytes} [Mon Oct 10 11:45:54 2022] GET /identity/v3/projects/55927449396d4c4f8470544dd7a23ed9 => generated 302 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:45:54.691742 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3598d9-d60d-414a-9285-71404129c3b0 None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:54.738030 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:54.799125 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3598d9-d60d-414a-9285-71404129c3b0 None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:54.810368 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1148de2-1da8-4d98-b402-844877e43fc9 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:54.838290 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891f72d8-9783-47bf-8eec-619d12ec79c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:54.839227 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891f72d8-9783-47bf-8eec-619d12ec79c6 None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:54.839755 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891f72d8-9783-47bf-8eec-619d12ec79c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:54.840174 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891f72d8-9783-47bf-8eec-619d12ec79c6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:54.841151 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-891f72d8-9783-47bf-8eec-619d12ec79c6 None admin] RBAC: Adding query filter params (role.id=0d822a0674b6495fac623877a3b38025, scope.project.id=55927449396d4c4f8470544dd7a23ed9, user.id=6c3317696b14428a8edb1085c6cfcc3d) {{(pid=81676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:45:54.841612 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-891f72d8-9783-47bf-8eec-619d12ec79c6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:54.843485 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-891f72d8-9783-47bf-8eec-619d12ec79c6 None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:54.848977 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc50a59-ed7e-4d72-bc88-6edee48b1230 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:54.849590 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc50a59-ed7e-4d72-bc88-6edee48b1230 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:54.849868 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc50a59-ed7e-4d72-bc88-6edee48b1230 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:54.850140 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc50a59-ed7e-4d72-bc88-6edee48b1230 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:54.850891 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc50a59-ed7e-4d72-bc88-6edee48b1230 None admin] RBAC: Adding query filter params (role.id=54c5cfe84bab471db7992eee92787387, scope.project.id=2d0af77a941540ef9b78b89ebae1ef1a, user.id=ae40cb146dee491b8706cc0d8a34df65) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:45:54.851199 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc50a59-ed7e-4d72-bc88-6edee48b1230 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:54.854725 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc50a59-ed7e-4d72-bc88-6edee48b1230 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:54.855198 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 91/206] 10.222.0.46 () {56 vars in 1374 bytes} [Mon Oct 10 11:45:54 2022] GET /identity/v3/role_assignments?user.id=6c3317696b14428a8edb1085c6cfcc3d&scope.project.id=55927449396d4c4f8470544dd7a23ed9&role.id=0d822a0674b6495fac623877a3b38025 => generated 259 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:45:54.856338 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891f72d8-9783-47bf-8eec-619d12ec79c6 None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:54.864794 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 116/207] 10.222.0.46 () {56 vars in 1374 bytes} [Mon Oct 10 11:45:54 2022] GET /identity/v3/role_assignments?user.id=ae40cb146dee491b8706cc0d8a34df65&scope.project.id=2d0af77a941540ef9b78b89ebae1ef1a&role.id=54c5cfe84bab471db7992eee92787387 => generated 608 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:45:54.933094 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:55.041097 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891f72d8-9783-47bf-8eec-619d12ec79c6 None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:55.113133 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a119461f-8436-40e9-8907-c84eb44cc954 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:55.113768 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a119461f-8436-40e9-8907-c84eb44cc954 None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:55.114043 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a119461f-8436-40e9-8907-c84eb44cc954 None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:55.114314 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a119461f-8436-40e9-8907-c84eb44cc954 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:55.115085 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a119461f-8436-40e9-8907-c84eb44cc954 None admin] RBAC: Adding query filter params (role.id=6762beebc2bf4e6f804cbbe55a475e70, scope.project.id=55927449396d4c4f8470544dd7a23ed9, user.id=6c3317696b14428a8edb1085c6cfcc3d) {{(pid=81676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:45:55.115401 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a119461f-8436-40e9-8907-c84eb44cc954 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:55.116883 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a119461f-8436-40e9-8907-c84eb44cc954 None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:55.131662 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 92/208] 10.222.0.46 () {56 vars in 1374 bytes} [Mon Oct 10 11:45:54 2022] GET /identity/v3/role_assignments?user.id=6c3317696b14428a8edb1085c6cfcc3d&scope.project.id=55927449396d4c4f8470544dd7a23ed9&role.id=6762beebc2bf4e6f804cbbe55a475e70 => generated 259 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:45:56.134303 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a7e8f4-ed8b-4e4a-8cfc-8ca8fbeb510c None None] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:56.134845 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a7e8f4-ed8b-4e4a-8cfc-8ca8fbeb510c None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:56.134920 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a7e8f4-ed8b-4e4a-8cfc-8ca8fbeb510c None None] PATH_INFO: `/` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:56.138648 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 117/209] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 10 11:45:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:45:56.138793 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4dd194-5f71-4155-b6a9-9777e8467d6d None None] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:56.138793 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4dd194-5f71-4155-b6a9-9777e8467d6d None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:56.138793 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4dd194-5f71-4155-b6a9-9777e8467d6d None None] PATH_INFO: `/` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:56.138793 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 93/210] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 10 11:45:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:45:56.141708 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341f6879-5b65-4ed1-88d2-bbcbe6b433a6 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:56.141997 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341f6879-5b65-4ed1-88d2-bbcbe6b433a6 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:56.142413 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341f6879-5b65-4ed1-88d2-bbcbe6b433a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:56.146395 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380c741e-94ab-469d-baa7-d600782c9f98 None None] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:56.146989 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380c741e-94ab-469d-baa7-d600782c9f98 None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:56.147520 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380c741e-94ab-469d-baa7-d600782c9f98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:56.161845 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-341f6879-5b65-4ed1-88d2-bbcbe6b433a6 None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:45:56.162804 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-341f6879-5b65-4ed1-88d2-bbcbe6b433a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:45:56.167699 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.auth.core [None req-380c741e-94ab-469d-baa7-d600782c9f98 None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:45:56.169095 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-380c741e-94ab-469d-baa7-d600782c9f98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:45:56.249172 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 118/211] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 10 11:45:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 10 11:45:56.255174 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 94/212] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 10 11:45:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 10 11:45:56.526519 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341f6879-5b65-4ed1-88d2-bbcbe6b433a6 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:56.534637 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380c741e-94ab-469d-baa7-d600782c9f98 None None] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:56.567859 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:56.599526 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:56.647393 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341f6879-5b65-4ed1-88d2-bbcbe6b433a6 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:56.664691 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380c741e-94ab-469d-baa7-d600782c9f98 None None] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:56.685142 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a21fcd-31a3-497c-9849-2ad3cda9b565 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:56.686197 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a21fcd-31a3-497c-9849-2ad3cda9b565 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:56.686759 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a21fcd-31a3-497c-9849-2ad3cda9b565 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:56.687257 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a21fcd-31a3-497c-9849-2ad3cda9b565 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:56.699382 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a21fcd-31a3-497c-9849-2ad3cda9b565 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 10 11:45:56.706033 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application [None req-d5a21fcd-31a3-497c-9849-2ad3cda9b565 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 10 11:45:56.706033 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:45:56.706033 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:45:56.706033 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:45:56.706033 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:45:56.706033 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:45:56.706033 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:45:56.706033 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:45:56.706033 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:45:56.706033 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:45:56.706033 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:45:56.706033 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:45:56.706033 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 10 11:45:56.706033 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 10 11:45:56.706033 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 10 11:45:56.706033 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 10 11:45:56.706966 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 10 11:45:56.706966 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 10 11:45:56.706966 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 10 11:45:56.706966 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 10 11:45:56.706966 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:45:56.706966 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:45:56.706966 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:45:56.706966 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:45:56.706966 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:45:56.706966 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:45:56.706966 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:45:56.706966 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application with Lock( Oct 10 11:45:56.706966 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:45:56.706966 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:45:56.706966 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:45:56.706966 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:45:56.707735 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:45:56.707735 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:45:56.707735 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:45:56.707735 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:45:56.707735 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 10 11:45:56.707735 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 10 11:45:56.707735 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 10 11:45:56.707735 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 10 11:45:56.707735 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 10 11:45:56.707735 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 10 11:45:56.707735 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 10 11:45:56.707735 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Oct 10 11:45:56.708322 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941f83e7-5532-400f-92e4-bccc5702ed84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:56.708322 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941f83e7-5532-400f-92e4-bccc5702ed84 None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:56.708322 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941f83e7-5532-400f-92e4-bccc5702ed84 None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:56.708322 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941f83e7-5532-400f-92e4-bccc5702ed84 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:56.709744 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941f83e7-5532-400f-92e4-bccc5702ed84 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:56.710554 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 119/213] 10.222.0.46 () {56 vars in 1099 bytes} [Mon Oct 10 11:45:56 2022] GET /identity/v3/roles/member => generated 84 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:45:56.715559 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941f83e7-5532-400f-92e4-bccc5702ed84 None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:56.717317 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 95/214] 10.222.0.46 () {56 vars in 1108 bytes} [Mon Oct 10 11:45:56 2022] GET /identity/v3/domains/default => generated 199 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 11:45:56.718699 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a21fcd-31a3-497c-9849-2ad3cda9b565 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:56.723932 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941f83e7-5532-400f-92e4-bccc5702ed84 None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:56.833677 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941f83e7-5532-400f-92e4-bccc5702ed84 None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:56.837663 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:56.848770 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a21fcd-31a3-497c-9849-2ad3cda9b565 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:56.856686 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:56.874002 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88505b81-b0ea-4f57-8f0f-399ffd0a2274 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:56.875021 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88505b81-b0ea-4f57-8f0f-399ffd0a2274 None admin] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:56.875473 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88505b81-b0ea-4f57-8f0f-399ffd0a2274 None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:56.876011 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88505b81-b0ea-4f57-8f0f-399ffd0a2274 None admin] PATH_INFO: `/v3/users` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:56.876934 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88505b81-b0ea-4f57-8f0f-399ffd0a2274 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:56.878595 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88505b81-b0ea-4f57-8f0f-399ffd0a2274 None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:56.891586 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78136d4-679d-409c-8d3b-75e093a32370 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:56.892936 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78136d4-679d-409c-8d3b-75e093a32370 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:56.893366 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78136d4-679d-409c-8d3b-75e093a32370 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:56.893935 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78136d4-679d-409c-8d3b-75e093a32370 None admin] PATH_INFO: `/v3/roles` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:56.894618 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78136d4-679d-409c-8d3b-75e093a32370 None admin] RBAC: Adding query filter params (name=member) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:45:56.894984 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78136d4-679d-409c-8d3b-75e093a32370 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:56.896409 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78136d4-679d-409c-8d3b-75e093a32370 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:56.905327 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 120/215] 10.222.0.46 () {56 vars in 1101 bytes} [Mon Oct 10 11:45:56 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:45:56.911316 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78136d4-679d-409c-8d3b-75e093a32370 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:56.957753 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 96/216] 10.222.0.46 () {60 vars in 1132 bytes} [Mon Oct 10 11:45:56 2022] POST /identity/v3/users => generated 297 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 10 11:45:57.050692 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78136d4-679d-409c-8d3b-75e093a32370 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:57.092176 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f13940a-851c-422b-afd1-95337ecb214c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:57.093275 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f13940a-851c-422b-afd1-95337ecb214c None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:57.093766 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f13940a-851c-422b-afd1-95337ecb214c None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:57.094459 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f13940a-851c-422b-afd1-95337ecb214c None admin] PATH_INFO: `/v3/users/3d151593b3ff4f6aa2d2009f3b4aba81` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:57.114892 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f13940a-851c-422b-afd1-95337ecb214c None admin] RBAC: Authorizing `identity:get_user(user_id=3d151593b3ff4f6aa2d2009f3b4aba81)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:57.116882 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f13940a-851c-422b-afd1-95337ecb214c None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:57.119088 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 121/217] 10.222.0.46 () {56 vars in 1177 bytes} [Mon Oct 10 11:45:56 2022] GET /identity/v3/users/3d151593b3ff4f6aa2d2009f3b4aba81 => generated 297 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 11:45:57.128553 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88505b81-b0ea-4f57-8f0f-399ffd0a2274 None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:57.214828 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30695ad0-6231-4e68-a938-7b80ecaf1929 None None] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:57.215333 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30695ad0-6231-4e68-a938-7b80ecaf1929 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:57.215407 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30695ad0-6231-4e68-a938-7b80ecaf1929 None None] PATH_INFO: `/` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:57.216711 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 122/218] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 10 11:45:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:45:57.222862 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac3c532-4e59-4043-8832-7c8aefd93813 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:57.223146 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac3c532-4e59-4043-8832-7c8aefd93813 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:57.223424 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac3c532-4e59-4043-8832-7c8aefd93813 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:57.247768 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-4ac3c532-4e59-4043-8832-7c8aefd93813 None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:45:57.250041 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-4ac3c532-4e59-4043-8832-7c8aefd93813 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:45:57.252219 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88505b81-b0ea-4f57-8f0f-399ffd0a2274 None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:57.259410 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:57.294912 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ec5596-85cc-4c24-bcbe-6161a8772f85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:57.295582 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ec5596-85cc-4c24-bcbe-6161a8772f85 None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:57.295869 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ec5596-85cc-4c24-bcbe-6161a8772f85 None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:57.297836 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ec5596-85cc-4c24-bcbe-6161a8772f85 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:57.337508 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 123/219] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 10 11:45:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 10 11:45:57.354349 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:57.379586 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ec5596-85cc-4c24-bcbe-6161a8772f85 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:57.381059 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ec5596-85cc-4c24-bcbe-6161a8772f85 None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:57.500714 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 97/220] 10.222.0.46 () {58 vars in 1282 bytes} [Mon Oct 10 11:45:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 10 11:45:57.506284 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac3c532-4e59-4043-8832-7c8aefd93813 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:57.561509 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9a32d2-73cf-4565-837f-a3a1398c6fcd None None] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:57.561893 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9a32d2-73cf-4565-837f-a3a1398c6fcd None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:57.562061 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9a32d2-73cf-4565-837f-a3a1398c6fcd None None] PATH_INFO: `/` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:57.563742 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 98/221] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 10 11:45:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:45:57.569188 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34006b0c-8468-4999-8b53-aa3fa8088397 None None] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:57.569479 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34006b0c-8468-4999-8b53-aa3fa8088397 None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:57.570054 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34006b0c-8468-4999-8b53-aa3fa8088397 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:57.600983 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.auth.core [None req-34006b0c-8468-4999-8b53-aa3fa8088397 None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:45:57.607410 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-34006b0c-8468-4999-8b53-aa3fa8088397 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:45:57.647948 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac3c532-4e59-4043-8832-7c8aefd93813 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:57.689685 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83c1a04-c3c5-4707-8fbf-73cefc3cc073 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:57.690352 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83c1a04-c3c5-4707-8fbf-73cefc3cc073 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:57.690631 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83c1a04-c3c5-4707-8fbf-73cefc3cc073 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:57.690936 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83c1a04-c3c5-4707-8fbf-73cefc3cc073 None admin] PATH_INFO: `/v3/projects/55927449396d4c4f8470544dd7a23ed9` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:57.692453 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83c1a04-c3c5-4707-8fbf-73cefc3cc073 None admin] RBAC: Authorizing `identity:get_project(project_id=55927449396d4c4f8470544dd7a23ed9)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:57.694142 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83c1a04-c3c5-4707-8fbf-73cefc3cc073 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:57.696162 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 124/222] 10.222.0.46 () {56 vars in 1186 bytes} [Mon Oct 10 11:45:57 2022] GET /identity/v3/projects/55927449396d4c4f8470544dd7a23ed9 => generated 302 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:45:57.697326 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83c1a04-c3c5-4707-8fbf-73cefc3cc073 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:57.702795 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 99/223] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 10 11:45:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 10 11:45:57.703908 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34006b0c-8468-4999-8b53-aa3fa8088397 None None] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:57.715294 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:57.722314 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:57.824008 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34006b0c-8468-4999-8b53-aa3fa8088397 None None] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:57.865112 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc5e1af-adf4-42f4-82a9-4cad57b34371 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:57.866263 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc5e1af-adf4-42f4-82a9-4cad57b34371 None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:57.866777 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc5e1af-adf4-42f4-82a9-4cad57b34371 None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:57.867564 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc5e1af-adf4-42f4-82a9-4cad57b34371 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:57.869464 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc5e1af-adf4-42f4-82a9-4cad57b34371 None admin] RBAC: Adding query filter params (role.id=b2cb2a6e818e47338ce071e6fedfc673, scope.project.id=55927449396d4c4f8470544dd7a23ed9, user.id=3d151593b3ff4f6aa2d2009f3b4aba81) {{(pid=81676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:45:57.870003 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc5e1af-adf4-42f4-82a9-4cad57b34371 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:57.871685 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc5e1af-adf4-42f4-82a9-4cad57b34371 None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:57.878597 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 100/224] 10.222.0.46 () {56 vars in 1374 bytes} [Mon Oct 10 11:45:57 2022] GET /identity/v3/role_assignments?user.id=3d151593b3ff4f6aa2d2009f3b4aba81&scope.project.id=55927449396d4c4f8470544dd7a23ed9&role.id=b2cb2a6e818e47338ce071e6fedfc673 => generated 259 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:45:57.927343 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83c1a04-c3c5-4707-8fbf-73cefc3cc073 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:57.986366 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cef355-1bc9-4eb7-b901-1ae7a73f173a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:57.987059 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cef355-1bc9-4eb7-b901-1ae7a73f173a None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:57.987383 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cef355-1bc9-4eb7-b901-1ae7a73f173a None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:57.987694 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cef355-1bc9-4eb7-b901-1ae7a73f173a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:58.004666 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49cef355-1bc9-4eb7-b901-1ae7a73f173a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 10 11:45:58.010108 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application [None req-49cef355-1bc9-4eb7-b901-1ae7a73f173a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 10 11:45:58.010108 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:45:58.010108 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:45:58.010108 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:45:58.010108 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:45:58.010108 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:45:58.010108 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:45:58.010108 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:45:58.010108 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:45:58.010108 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:45:58.010108 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:45:58.010108 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:45:58.010108 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 10 11:45:58.010108 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 10 11:45:58.010108 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 10 11:45:58.010108 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 10 11:45:58.011044 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 10 11:45:58.011044 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 10 11:45:58.011044 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 10 11:45:58.011044 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 10 11:45:58.011044 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:45:58.011044 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:45:58.011044 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:45:58.011044 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:45:58.011044 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:45:58.011044 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:45:58.011044 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:45:58.011044 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application with Lock( Oct 10 11:45:58.011044 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:45:58.011044 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:45:58.011044 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:45:58.011044 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:45:58.011814 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:45:58.011814 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:45:58.011814 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:45:58.011814 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:45:58.011814 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 10 11:45:58.011814 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 10 11:45:58.011814 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 10 11:45:58.011814 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 10 11:45:58.011814 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 10 11:45:58.011814 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 10 11:45:58.011814 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 10 11:45:58.011814 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Oct 10 11:45:58.011814 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 125/225] 10.222.0.46 () {56 vars in 1096 bytes} [Mon Oct 10 11:45:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 315 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:45:58.017820 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc5e1af-adf4-42f4-82a9-4cad57b34371 None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:58.041202 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cef355-1bc9-4eb7-b901-1ae7a73f173a None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:58.139061 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:58.164695 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:58.176264 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cef355-1bc9-4eb7-b901-1ae7a73f173a None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:58.229360 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c68767-6d84-4bf0-8131-151101e2ce0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:58.230856 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c68767-6d84-4bf0-8131-151101e2ce0d None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:58.232045 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c68767-6d84-4bf0-8131-151101e2ce0d None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:58.232524 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c68767-6d84-4bf0-8131-151101e2ce0d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:58.243826 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc5e1af-adf4-42f4-82a9-4cad57b34371 None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:58.257479 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c68767-6d84-4bf0-8131-151101e2ce0d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 10 11:45:58.267033 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application [None req-23c68767-6d84-4bf0-8131-151101e2ce0d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 10 11:45:58.267033 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:45:58.267033 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:45:58.267033 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:45:58.267033 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:45:58.267033 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:45:58.267033 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:45:58.267033 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:45:58.267033 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:45:58.267033 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:45:58.267033 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:45:58.267033 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:45:58.267033 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 10 11:45:58.267033 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 10 11:45:58.267033 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 10 11:45:58.267033 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 10 11:45:58.268384 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 10 11:45:58.268384 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 10 11:45:58.268384 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 10 11:45:58.268384 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 10 11:45:58.268384 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:45:58.268384 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:45:58.268384 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:45:58.268384 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:45:58.268384 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:45:58.268384 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:45:58.268384 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:45:58.268384 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application with Lock( Oct 10 11:45:58.268384 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:45:58.268384 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:45:58.268384 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:45:58.268384 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:45:58.269147 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:45:58.269147 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:45:58.269147 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:45:58.269147 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:45:58.269147 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 10 11:45:58.269147 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 10 11:45:58.269147 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 10 11:45:58.269147 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 10 11:45:58.269147 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 10 11:45:58.269147 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 10 11:45:58.269147 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 10 11:45:58.269147 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Oct 10 11:45:58.269147 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 126/226] 10.222.0.46 () {56 vars in 1114 bytes} [Mon Oct 10 11:45:58 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:45:58.274421 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c68767-6d84-4bf0-8131-151101e2ce0d None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:58.311286 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909dcdec-1110-41de-8c62-f6ef3f195238 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:58.311949 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909dcdec-1110-41de-8c62-f6ef3f195238 None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:58.312226 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909dcdec-1110-41de-8c62-f6ef3f195238 None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:58.312767 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909dcdec-1110-41de-8c62-f6ef3f195238 None admin] PATH_INFO: `/v3/roles` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:58.313272 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909dcdec-1110-41de-8c62-f6ef3f195238 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:45:58.313583 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909dcdec-1110-41de-8c62-f6ef3f195238 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:58.315062 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909dcdec-1110-41de-8c62-f6ef3f195238 None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:58.334700 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 101/227] 10.222.0.46 () {56 vars in 1099 bytes} [Mon Oct 10 11:45:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 318 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:45:58.340401 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909dcdec-1110-41de-8c62-f6ef3f195238 None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:58.410006 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c68767-6d84-4bf0-8131-151101e2ce0d None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:58.435180 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:58.451507 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0b6d90-7537-41ac-8838-63bc774964e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:58.452323 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0b6d90-7537-41ac-8838-63bc774964e9 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:58.452602 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0b6d90-7537-41ac-8838-63bc774964e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:58.452886 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0b6d90-7537-41ac-8838-63bc774964e9 None admin] PATH_INFO: `/v3/roles` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:58.453592 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0b6d90-7537-41ac-8838-63bc774964e9 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:45:58.453898 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0b6d90-7537-41ac-8838-63bc774964e9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:58.458939 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0b6d90-7537-41ac-8838-63bc774964e9 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:58.474144 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 127/228] 10.222.0.46 () {56 vars in 1111 bytes} [Mon Oct 10 11:45:58 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 11:45:58.482375 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0b6d90-7537-41ac-8838-63bc774964e9 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:58.552115 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909dcdec-1110-41de-8c62-f6ef3f195238 None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:58.630934 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0b6d90-7537-41ac-8838-63bc774964e9 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:58.667115 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:58.667491 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fabb3d-3ea2-4e59-89d9-dcb6ef7372f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:58.668228 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fabb3d-3ea2-4e59-89d9-dcb6ef7372f5 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:58.668514 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fabb3d-3ea2-4e59-89d9-dcb6ef7372f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:58.668799 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fabb3d-3ea2-4e59-89d9-dcb6ef7372f5 None admin] PATH_INFO: `/v3/users/6c3317696b14428a8edb1085c6cfcc3d` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:58.670013 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90fabb3d-3ea2-4e59-89d9-dcb6ef7372f5 None admin] RBAC: Authorizing `identity:get_user(user_id=6c3317696b14428a8edb1085c6cfcc3d)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:58.671631 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90fabb3d-3ea2-4e59-89d9-dcb6ef7372f5 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:58.673255 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 128/229] 10.222.0.46 () {56 vars in 1177 bytes} [Mon Oct 10 11:45:58 2022] GET /identity/v3/users/6c3317696b14428a8edb1085c6cfcc3d => generated 283 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:45:58.674063 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c72805-0689-41c5-9740-168d16037dbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:58.674754 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c72805-0689-41c5-9740-168d16037dbd None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:58.675044 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c72805-0689-41c5-9740-168d16037dbd None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:58.675325 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c72805-0689-41c5-9740-168d16037dbd None admin] PATH_INFO: `/v3/users/6c3317696b14428a8edb1085c6cfcc3d` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:58.681659 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c72805-0689-41c5-9740-168d16037dbd None admin] RBAC: Authorizing `identity:get_user(user_id=6c3317696b14428a8edb1085c6cfcc3d)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:58.681778 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fabb3d-3ea2-4e59-89d9-dcb6ef7372f5 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:58.681960 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c72805-0689-41c5-9740-168d16037dbd None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:58.681960 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 102/230] 10.222.0.46 () {56 vars in 1177 bytes} [Mon Oct 10 11:45:58 2022] GET /identity/v3/users/6c3317696b14428a8edb1085c6cfcc3d => generated 283 bytes in 342 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 11:45:58.693635 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c72805-0689-41c5-9740-168d16037dbd None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:58.746728 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:58.791549 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fabb3d-3ea2-4e59-89d9-dcb6ef7372f5 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:58.828538 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec27e5e-f2dd-438c-87a7-9699b7998a4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:58.829329 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec27e5e-f2dd-438c-87a7-9699b7998a4c None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:58.829739 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec27e5e-f2dd-438c-87a7-9699b7998a4c None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:58.830109 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec27e5e-f2dd-438c-87a7-9699b7998a4c None admin] PATH_INFO: `/v3/projects/55927449396d4c4f8470544dd7a23ed9` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:58.831367 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec27e5e-f2dd-438c-87a7-9699b7998a4c None admin] RBAC: Authorizing `identity:get_project(project_id=55927449396d4c4f8470544dd7a23ed9)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:58.832926 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec27e5e-f2dd-438c-87a7-9699b7998a4c None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:58.835136 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 129/231] 10.222.0.46 () {56 vars in 1186 bytes} [Mon Oct 10 11:45:58 2022] GET /identity/v3/projects/55927449396d4c4f8470544dd7a23ed9 => generated 302 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:45:58.840801 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec27e5e-f2dd-438c-87a7-9699b7998a4c None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:58.949838 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec27e5e-f2dd-438c-87a7-9699b7998a4c None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:58.952238 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c72805-0689-41c5-9740-168d16037dbd None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:58.979237 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:58.987677 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b9dfbf-e1d4-45bd-8270-681a4ac93bbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:58.992827 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b9dfbf-e1d4-45bd-8270-681a4ac93bbb None admin] REQUEST_METHOD: `PUT` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:58.992827 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b9dfbf-e1d4-45bd-8270-681a4ac93bbb None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:58.992827 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b9dfbf-e1d4-45bd-8270-681a4ac93bbb None admin] PATH_INFO: `/v3/projects/55927449396d4c4f8470544dd7a23ed9/users/6c3317696b14428a8edb1085c6cfcc3d/roles/6762beebc2bf4e6f804cbbe55a475e70` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:58.992827 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b9dfbf-e1d4-45bd-8270-681a4ac93bbb None admin] RBAC: Authorizing `identity:create_grant(project_id=55927449396d4c4f8470544dd7a23ed9, user_id=6c3317696b14428a8edb1085c6cfcc3d, role_id=6762beebc2bf4e6f804cbbe55a475e70)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:58.993152 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b9dfbf-e1d4-45bd-8270-681a4ac93bbb None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:59.018843 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7849d055-9724-47ed-af29-59cb2455f4ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:59.019749 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7849d055-9724-47ed-af29-59cb2455f4ea None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:59.020208 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7849d055-9724-47ed-af29-59cb2455f4ea None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:59.020619 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7849d055-9724-47ed-af29-59cb2455f4ea None admin] PATH_INFO: `/v3/projects/55927449396d4c4f8470544dd7a23ed9` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:59.021874 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7849d055-9724-47ed-af29-59cb2455f4ea None admin] RBAC: Authorizing `identity:get_project(project_id=55927449396d4c4f8470544dd7a23ed9)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:59.023520 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7849d055-9724-47ed-af29-59cb2455f4ea None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:59.025361 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 103/232] 10.222.0.46 () {56 vars in 1186 bytes} [Mon Oct 10 11:45:58 2022] GET /identity/v3/projects/55927449396d4c4f8470544dd7a23ed9 => generated 302 bytes in 332 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:45:59.033228 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7849d055-9724-47ed-af29-59cb2455f4ea None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:45:59.080507 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 130/233] 10.222.0.46 () {58 vars in 1439 bytes} [Mon Oct 10 11:45:58 2022] PUT /identity/v3/projects/55927449396d4c4f8470544dd7a23ed9/users/6c3317696b14428a8edb1085c6cfcc3d/roles/6762beebc2bf4e6f804cbbe55a475e70 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 10 11:45:59.136245 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:45:59.188196 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7849d055-9724-47ed-af29-59cb2455f4ea None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:45:59.243971 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9decec1e-dc3d-42a9-a3e1-12eeb6f0785b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:45:59.244790 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9decec1e-dc3d-42a9-a3e1-12eeb6f0785b None admin] REQUEST_METHOD: `PUT` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:59.245097 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9decec1e-dc3d-42a9-a3e1-12eeb6f0785b None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:59.245377 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9decec1e-dc3d-42a9-a3e1-12eeb6f0785b None admin] PATH_INFO: `/v3/projects/55927449396d4c4f8470544dd7a23ed9/users/6c3317696b14428a8edb1085c6cfcc3d/roles/0d822a0674b6495fac623877a3b38025` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:59.247611 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9decec1e-dc3d-42a9-a3e1-12eeb6f0785b None admin] RBAC: Authorizing `identity:create_grant(project_id=55927449396d4c4f8470544dd7a23ed9, user_id=6c3317696b14428a8edb1085c6cfcc3d, role_id=0d822a0674b6495fac623877a3b38025)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:45:59.249407 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9decec1e-dc3d-42a9-a3e1-12eeb6f0785b None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:45:59.333207 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 104/234] 10.222.0.46 () {58 vars in 1439 bytes} [Mon Oct 10 11:45:59 2022] PUT /identity/v3/projects/55927449396d4c4f8470544dd7a23ed9/users/6c3317696b14428a8edb1085c6cfcc3d/roles/0d822a0674b6495fac623877a3b38025 => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 10 11:45:59.868545 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe57895-66bb-4d17-b634-bdae64cd237d None None] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:59.869053 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe57895-66bb-4d17-b634-bdae64cd237d None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:59.869162 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe57895-66bb-4d17-b634-bdae64cd237d None None] PATH_INFO: `/` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:59.870681 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 131/235] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 10 11:45:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:45:59.877194 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7aa922b-2ee9-4fa2-9ecc-542dddcda231 None None] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:59.877483 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7aa922b-2ee9-4fa2-9ecc-542dddcda231 None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:59.877764 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7aa922b-2ee9-4fa2-9ecc-542dddcda231 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:59.903729 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.auth.core [None req-b7aa922b-2ee9-4fa2-9ecc-542dddcda231 None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:45:59.904939 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-b7aa922b-2ee9-4fa2-9ecc-542dddcda231 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:45:59.916051 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544cfe02-40c9-4da0-98bf-c02d84ced1c0 None None] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:59.917032 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544cfe02-40c9-4da0-98bf-c02d84ced1c0 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:59.917828 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544cfe02-40c9-4da0-98bf-c02d84ced1c0 None None] PATH_INFO: `/` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:59.919737 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 132/236] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 10 11:45:59 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 10 11:45:59.927945 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4c8156-2dad-4b86-94e8-a31589d059bb None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:45:59.928756 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4c8156-2dad-4b86-94e8-a31589d059bb None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:45:59.929248 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4c8156-2dad-4b86-94e8-a31589d059bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:45:59.949034 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-da4c8156-2dad-4b86-94e8-a31589d059bb None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:45:59.950568 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-da4c8156-2dad-4b86-94e8-a31589d059bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:46:00.018284 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 105/237] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 10 11:45:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 10 11:46:00.027188 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 133/238] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 10 11:45:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 10 11:46:00.293884 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7aa922b-2ee9-4fa2-9ecc-542dddcda231 None None] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:00.374466 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:00.397242 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4c8156-2dad-4b86-94e8-a31589d059bb None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:00.450899 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7aa922b-2ee9-4fa2-9ecc-542dddcda231 None None] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:00.518952 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5d9489-4c8b-459e-b17e-f7beb33a8a5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:00.519615 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5d9489-4c8b-459e-b17e-f7beb33a8a5b None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:00.519893 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5d9489-4c8b-459e-b17e-f7beb33a8a5b None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:00.520164 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5d9489-4c8b-459e-b17e-f7beb33a8a5b None admin] PATH_INFO: `/v3/domains/default` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:00.522555 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5d9489-4c8b-459e-b17e-f7beb33a8a5b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:00.524098 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5d9489-4c8b-459e-b17e-f7beb33a8a5b None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:00.526016 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 106/239] 10.222.0.46 () {56 vars in 1108 bytes} [Mon Oct 10 11:46:00 2022] GET /identity/v3/domains/default => generated 199 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:46:00.539686 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5d9489-4c8b-459e-b17e-f7beb33a8a5b None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:00.566834 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4c8156-2dad-4b86-94e8-a31589d059bb None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:00.587782 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:00.605089 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f10955-0f8e-4c05-8466-851938e1156f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:00.605740 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f10955-0f8e-4c05-8466-851938e1156f None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:00.606065 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f10955-0f8e-4c05-8466-851938e1156f None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:00.606353 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f10955-0f8e-4c05-8466-851938e1156f None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:00.617202 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f10955-0f8e-4c05-8466-851938e1156f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 10 11:46:00.622233 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application [None req-53f10955-0f8e-4c05-8466-851938e1156f None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 10 11:46:00.622233 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:46:00.622233 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:46:00.622233 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:46:00.622233 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:46:00.622233 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:46:00.622233 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:46:00.622233 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:46:00.622233 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:46:00.622233 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:46:00.622233 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:46:00.622233 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:46:00.622233 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 10 11:46:00.622233 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 10 11:46:00.622233 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 10 11:46:00.622233 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 10 11:46:00.623403 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 10 11:46:00.623403 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 10 11:46:00.623403 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 10 11:46:00.623403 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 10 11:46:00.623403 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:46:00.623403 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:46:00.623403 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:46:00.623403 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:46:00.623403 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:46:00.623403 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:46:00.623403 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:46:00.623403 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application with Lock( Oct 10 11:46:00.623403 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:46:00.623403 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:46:00.623403 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:46:00.623403 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:46:00.624182 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:46:00.624182 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:46:00.624182 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:46:00.624182 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:46:00.624182 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 10 11:46:00.624182 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 10 11:46:00.624182 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 10 11:46:00.624182 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 10 11:46:00.624182 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 10 11:46:00.624182 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 10 11:46:00.624182 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 10 11:46:00.624182 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Oct 10 11:46:00.624182 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 134/240] 10.222.0.46 () {56 vars in 1099 bytes} [Mon Oct 10 11:46:00 2022] GET /identity/v3/roles/reader => generated 84 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:46:00.636227 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f10955-0f8e-4c05-8466-851938e1156f None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:00.721828 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5d9489-4c8b-459e-b17e-f7beb33a8a5b None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:00.763724 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:00.763911 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f10955-0f8e-4c05-8466-851938e1156f None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:00.780107 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aac2e48-05a6-4e7d-a7bc-387d266b2274 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:00.781271 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aac2e48-05a6-4e7d-a7bc-387d266b2274 None admin] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:00.781780 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aac2e48-05a6-4e7d-a7bc-387d266b2274 None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:00.782530 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aac2e48-05a6-4e7d-a7bc-387d266b2274 None admin] PATH_INFO: `/v3/users` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:00.784261 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aac2e48-05a6-4e7d-a7bc-387d266b2274 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:00.785984 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aac2e48-05a6-4e7d-a7bc-387d266b2274 None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:00.802546 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7025ae-99de-4bba-973d-5cfed69ea6ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:00.803359 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7025ae-99de-4bba-973d-5cfed69ea6ac None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:00.803773 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7025ae-99de-4bba-973d-5cfed69ea6ac None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:00.804246 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7025ae-99de-4bba-973d-5cfed69ea6ac None admin] PATH_INFO: `/v3/roles` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:00.805124 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7025ae-99de-4bba-973d-5cfed69ea6ac None admin] RBAC: Adding query filter params (name=reader) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:46:00.805488 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7025ae-99de-4bba-973d-5cfed69ea6ac None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:00.807143 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7025ae-99de-4bba-973d-5cfed69ea6ac None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:00.816640 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 135/241] 10.222.0.46 () {56 vars in 1101 bytes} [Mon Oct 10 11:46:00 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:46:00.818092 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ed1411-bae5-4256-87c7-8efa2084de67 None None] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:00.818378 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ed1411-bae5-4256-87c7-8efa2084de67 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:00.818738 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ed1411-bae5-4256-87c7-8efa2084de67 None None] PATH_INFO: `/` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:00.820122 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 136/242] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 10 11:46:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:46:00.823544 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ed1411-bae5-4256-87c7-8efa2084de67 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:00.878596 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:00.881260 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 107/243] 10.222.0.46 () {60 vars in 1132 bytes} [Mon Oct 10 11:46:00 2022] POST /identity/v3/users => generated 293 bytes in 342 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 11:46:00.882871 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fddd33b-b7b0-45f0-aa50-8ebc79e89046 None None] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:00.883369 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fddd33b-b7b0-45f0-aa50-8ebc79e89046 None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:00.883866 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fddd33b-b7b0-45f0-aa50-8ebc79e89046 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:00.914992 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.auth.core [None req-4fddd33b-b7b0-45f0-aa50-8ebc79e89046 None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:46:00.916523 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-4fddd33b-b7b0-45f0-aa50-8ebc79e89046 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:46:01.004267 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ed1411-bae5-4256-87c7-8efa2084de67 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:01.073473 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befa875b-c9e7-4f5c-8125-68ef61ba90d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:01.074058 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befa875b-c9e7-4f5c-8125-68ef61ba90d3 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:01.074344 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befa875b-c9e7-4f5c-8125-68ef61ba90d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:01.074622 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befa875b-c9e7-4f5c-8125-68ef61ba90d3 None admin] PATH_INFO: `/v3/users/d2da0904dbe64369af873368dbed2810` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:01.086889 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 108/244] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 10 11:46:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 10 11:46:01.107907 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-befa875b-c9e7-4f5c-8125-68ef61ba90d3 None admin] RBAC: Authorizing `identity:get_user(user_id=d2da0904dbe64369af873368dbed2810)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:01.109403 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-befa875b-c9e7-4f5c-8125-68ef61ba90d3 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:01.111712 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 137/245] 10.222.0.46 () {56 vars in 1177 bytes} [Mon Oct 10 11:46:00 2022] GET /identity/v3/users/d2da0904dbe64369af873368dbed2810 => generated 297 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:46:01.116888 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fddd33b-b7b0-45f0-aa50-8ebc79e89046 None None] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:01.233095 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fddd33b-b7b0-45f0-aa50-8ebc79e89046 None None] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:01.247223 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:01.270617 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35639845-3ba8-4292-86b8-e45a7dc1f6af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:01.271269 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35639845-3ba8-4292-86b8-e45a7dc1f6af None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:01.271551 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35639845-3ba8-4292-86b8-e45a7dc1f6af None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:01.271826 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35639845-3ba8-4292-86b8-e45a7dc1f6af None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:01.344353 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35639845-3ba8-4292-86b8-e45a7dc1f6af None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:01.345805 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35639845-3ba8-4292-86b8-e45a7dc1f6af None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:01.435051 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befa875b-c9e7-4f5c-8125-68ef61ba90d3 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:01.469605 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 109/246] 10.222.0.46 () {58 vars in 1282 bytes} [Mon Oct 10 11:46:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 10 11:46:01.479824 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35639845-3ba8-4292-86b8-e45a7dc1f6af None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:01.507093 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:01.586686 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:01.591625 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befa875b-c9e7-4f5c-8125-68ef61ba90d3 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:01.619757 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35639845-3ba8-4292-86b8-e45a7dc1f6af None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:01.651435 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978779bb-9438-4ab9-ba2d-24e038f665e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:01.652114 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978779bb-9438-4ab9-ba2d-24e038f665e6 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:01.652399 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978779bb-9438-4ab9-ba2d-24e038f665e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:01.653003 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978779bb-9438-4ab9-ba2d-24e038f665e6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:01.670594 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978779bb-9438-4ab9-ba2d-24e038f665e6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 10 11:46:01.672819 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080b90ff-df50-47e2-a48b-3999338c9eb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:01.673534 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080b90ff-df50-47e2-a48b-3999338c9eb6 None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:01.673818 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080b90ff-df50-47e2-a48b-3999338c9eb6 None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:01.674097 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080b90ff-df50-47e2-a48b-3999338c9eb6 None admin] PATH_INFO: `/v3/projects/55927449396d4c4f8470544dd7a23ed9` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:01.675430 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080b90ff-df50-47e2-a48b-3999338c9eb6 None admin] RBAC: Authorizing `identity:get_project(project_id=55927449396d4c4f8470544dd7a23ed9)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:01.676377 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application [None req-978779bb-9438-4ab9-ba2d-24e038f665e6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 10 11:46:01.676377 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:46:01.676377 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:46:01.676377 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:46:01.676377 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:46:01.676377 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:46:01.676377 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:46:01.676377 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:46:01.676377 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:46:01.676377 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:46:01.676377 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:46:01.676377 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:46:01.676377 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 10 11:46:01.676377 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 10 11:46:01.676377 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 10 11:46:01.676377 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 10 11:46:01.677297 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 10 11:46:01.677297 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 10 11:46:01.677297 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 10 11:46:01.677297 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 10 11:46:01.677297 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:46:01.677297 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:46:01.677297 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:46:01.677297 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:46:01.677297 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:46:01.677297 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:46:01.677297 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:46:01.677297 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application with Lock( Oct 10 11:46:01.677297 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:46:01.677297 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:46:01.677297 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:46:01.677297 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:46:01.678054 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:46:01.678054 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:46:01.678054 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:46:01.678054 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:46:01.678054 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 10 11:46:01.678054 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 10 11:46:01.678054 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 10 11:46:01.678054 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 10 11:46:01.678054 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 10 11:46:01.678054 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 10 11:46:01.678054 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 10 11:46:01.678054 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Oct 10 11:46:01.678636 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080b90ff-df50-47e2-a48b-3999338c9eb6 None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:01.678755 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 138/247] 10.222.0.46 () {56 vars in 1099 bytes} [Mon Oct 10 11:46:01 2022] GET /identity/v3/roles/member => generated 84 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 10 11:46:01.678821 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 110/248] 10.222.0.46 () {56 vars in 1186 bytes} [Mon Oct 10 11:46:01 2022] GET /identity/v3/projects/55927449396d4c4f8470544dd7a23ed9 => generated 302 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:46:01.686933 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080b90ff-df50-47e2-a48b-3999338c9eb6 None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:01.687254 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978779bb-9438-4ab9-ba2d-24e038f665e6 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:01.803910 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:01.816210 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978779bb-9438-4ab9-ba2d-24e038f665e6 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:01.825795 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080b90ff-df50-47e2-a48b-3999338c9eb6 None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:01.888087 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d284a396-e8ae-45c2-a640-e59eaa710b7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:01.888898 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d284a396-e8ae-45c2-a640-e59eaa710b7d None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:01.889064 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d284a396-e8ae-45c2-a640-e59eaa710b7d None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:01.889429 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d284a396-e8ae-45c2-a640-e59eaa710b7d None admin] PATH_INFO: `/v3/roles` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:01.889944 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d284a396-e8ae-45c2-a640-e59eaa710b7d None admin] RBAC: Adding query filter params (name=member) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:46:01.890252 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d284a396-e8ae-45c2-a640-e59eaa710b7d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:01.892888 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d284a396-e8ae-45c2-a640-e59eaa710b7d None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:01.909149 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 139/249] 10.222.0.46 () {56 vars in 1101 bytes} [Mon Oct 10 11:46:01 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:46:01.910568 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1b0703-cea7-4d11-a393-aef00e2d0820 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:01.911313 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1b0703-cea7-4d11-a393-aef00e2d0820 None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:01.911564 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1b0703-cea7-4d11-a393-aef00e2d0820 None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:01.911808 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1b0703-cea7-4d11-a393-aef00e2d0820 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:01.912525 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1b0703-cea7-4d11-a393-aef00e2d0820 None admin] RBAC: Adding query filter params (role.id=54c5cfe84bab471db7992eee92787387, scope.project.id=55927449396d4c4f8470544dd7a23ed9, user.id=d2da0904dbe64369af873368dbed2810) {{(pid=81676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:46:01.912839 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1b0703-cea7-4d11-a393-aef00e2d0820 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:01.915342 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b248f00-31e1-4372-a332-1355a2aa092c None None] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:01.915638 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b248f00-31e1-4372-a332-1355a2aa092c None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:01.915886 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b248f00-31e1-4372-a332-1355a2aa092c None None] PATH_INFO: `/` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:01.916839 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1b0703-cea7-4d11-a393-aef00e2d0820 None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:01.919650 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 140/250] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 10 11:46:01 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:46:01.922291 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b248f00-31e1-4372-a332-1355a2aa092c None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:01.927658 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 111/251] 10.222.0.46 () {56 vars in 1374 bytes} [Mon Oct 10 11:46:01 2022] GET /identity/v3/role_assignments?user.id=d2da0904dbe64369af873368dbed2810&scope.project.id=55927449396d4c4f8470544dd7a23ed9&role.id=54c5cfe84bab471db7992eee92787387 => generated 259 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:46:01.928911 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb033ca-1743-4a71-9bef-5918e061fcfc None None] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:01.929202 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb033ca-1743-4a71-9bef-5918e061fcfc None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:01.929461 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb033ca-1743-4a71-9bef-5918e061fcfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:01.965989 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.auth.core [None req-5fb033ca-1743-4a71-9bef-5918e061fcfc None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:46:01.967232 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-5fb033ca-1743-4a71-9bef-5918e061fcfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:46:02.006598 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:02.117222 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b248f00-31e1-4372-a332-1355a2aa092c None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:02.120508 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 112/252] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 10 11:46:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 10 11:46:02.121774 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a69a9c-d448-4245-b590-31aa2017b086 None None] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:02.122069 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a69a9c-d448-4245-b590-31aa2017b086 None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:02.122347 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a69a9c-d448-4245-b590-31aa2017b086 None None] PATH_INFO: `/` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:02.124042 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 113/253] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 10 11:46:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:46:02.131775 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa949803-9cd3-4cca-aa64-3460c79788e1 None None] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:02.132078 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa949803-9cd3-4cca-aa64-3460c79788e1 None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:02.132322 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa949803-9cd3-4cca-aa64-3460c79788e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:02.164014 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be829631-807f-412c-b6e5-1b00a8562227 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:02.164594 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be829631-807f-412c-b6e5-1b00a8562227 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:02.164876 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be829631-807f-412c-b6e5-1b00a8562227 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:02.165152 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be829631-807f-412c-b6e5-1b00a8562227 None admin] PATH_INFO: `/v3/users/3d151593b3ff4f6aa2d2009f3b4aba81` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:02.166378 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be829631-807f-412c-b6e5-1b00a8562227 None admin] RBAC: Authorizing `identity:get_user(user_id=3d151593b3ff4f6aa2d2009f3b4aba81)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:02.167894 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be829631-807f-412c-b6e5-1b00a8562227 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:02.170302 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 141/254] 10.222.0.46 () {56 vars in 1177 bytes} [Mon Oct 10 11:46:01 2022] GET /identity/v3/users/3d151593b3ff4f6aa2d2009f3b4aba81 => generated 297 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 11:46:02.176076 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.auth.core [None req-aa949803-9cd3-4cca-aa64-3460c79788e1 None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:46:02.176405 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be829631-807f-412c-b6e5-1b00a8562227 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:02.177432 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-aa949803-9cd3-4cca-aa64-3460c79788e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:46:02.336977 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 114/255] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 10 11:46:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 10 11:46:02.355105 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:02.365122 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be829631-807f-412c-b6e5-1b00a8562227 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:02.405820 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230f5125-cdc7-43f2-9715-d4a3eac8b898 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:02.406473 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230f5125-cdc7-43f2-9715-d4a3eac8b898 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:02.406769 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230f5125-cdc7-43f2-9715-d4a3eac8b898 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:02.407044 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230f5125-cdc7-43f2-9715-d4a3eac8b898 None admin] PATH_INFO: `/v3/projects/55927449396d4c4f8470544dd7a23ed9` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:02.408766 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230f5125-cdc7-43f2-9715-d4a3eac8b898 None admin] RBAC: Authorizing `identity:get_project(project_id=55927449396d4c4f8470544dd7a23ed9)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:02.410211 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230f5125-cdc7-43f2-9715-d4a3eac8b898 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:02.412078 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 142/256] 10.222.0.46 () {56 vars in 1186 bytes} [Mon Oct 10 11:46:02 2022] GET /identity/v3/projects/55927449396d4c4f8470544dd7a23ed9 => generated 302 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 11:46:02.417629 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa949803-9cd3-4cca-aa64-3460c79788e1 None None] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:02.421854 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:02.566042 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa949803-9cd3-4cca-aa64-3460c79788e1 None None] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:02.612328 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3150194d-dbf9-42b0-9248-388b71d04c55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:02.612999 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3150194d-dbf9-42b0-9248-388b71d04c55 None admin] REQUEST_METHOD: `PUT` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:02.613276 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3150194d-dbf9-42b0-9248-388b71d04c55 None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:02.613548 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3150194d-dbf9-42b0-9248-388b71d04c55 None admin] PATH_INFO: `/v3/projects/55927449396d4c4f8470544dd7a23ed9/users/3d151593b3ff4f6aa2d2009f3b4aba81/roles/b2cb2a6e818e47338ce071e6fedfc673` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:02.615520 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3150194d-dbf9-42b0-9248-388b71d04c55 None admin] RBAC: Authorizing `identity:create_grant(project_id=55927449396d4c4f8470544dd7a23ed9, user_id=3d151593b3ff4f6aa2d2009f3b4aba81, role_id=b2cb2a6e818e47338ce071e6fedfc673)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:02.616966 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3150194d-dbf9-42b0-9248-388b71d04c55 None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:02.652420 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230f5125-cdc7-43f2-9715-d4a3eac8b898 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:02.703894 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 115/257] 10.222.0.46 () {58 vars in 1439 bytes} [Mon Oct 10 11:46:02 2022] PUT /identity/v3/projects/55927449396d4c4f8470544dd7a23ed9/users/3d151593b3ff4f6aa2d2009f3b4aba81/roles/b2cb2a6e818e47338ce071e6fedfc673 => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 11:46:02.768323 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3150194d-dbf9-42b0-9248-388b71d04c55 None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:02.804252 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230f5125-cdc7-43f2-9715-d4a3eac8b898 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:02.857791 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a55d69-3662-4efd-a26b-5f82b222ff0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:02.859209 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a55d69-3662-4efd-a26b-5f82b222ff0f None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:02.859499 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a55d69-3662-4efd-a26b-5f82b222ff0f None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:02.859776 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a55d69-3662-4efd-a26b-5f82b222ff0f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:02.884188 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a55d69-3662-4efd-a26b-5f82b222ff0f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 10 11:46:02.889877 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application [None req-a7a55d69-3662-4efd-a26b-5f82b222ff0f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 10 11:46:02.889877 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:46:02.889877 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:46:02.889877 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:46:02.889877 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:46:02.889877 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:46:02.889877 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:46:02.889877 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:46:02.889877 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:46:02.889877 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:46:02.889877 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:46:02.889877 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:46:02.889877 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 10 11:46:02.889877 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 10 11:46:02.889877 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 10 11:46:02.889877 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 10 11:46:02.891284 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 10 11:46:02.891284 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 10 11:46:02.891284 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 10 11:46:02.891284 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 10 11:46:02.891284 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:46:02.891284 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:46:02.891284 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:46:02.891284 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:46:02.891284 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:46:02.891284 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:46:02.891284 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:46:02.891284 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application with Lock( Oct 10 11:46:02.891284 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:46:02.891284 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:46:02.891284 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:46:02.891284 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:46:02.892655 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:46:02.892655 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:46:02.892655 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:46:02.892655 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:46:02.892655 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 10 11:46:02.892655 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 10 11:46:02.892655 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 10 11:46:02.892655 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 10 11:46:02.892655 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 10 11:46:02.892655 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 10 11:46:02.892655 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 10 11:46:02.892655 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Oct 10 11:46:02.898246 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 143/258] 10.222.0.46 () {56 vars in 1114 bytes} [Mon Oct 10 11:46:02 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 10 11:46:02.899618 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a55d69-3662-4efd-a26b-5f82b222ff0f None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:02.918485 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:02.955276 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:02.981765 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3150194d-dbf9-42b0-9248-388b71d04c55 None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:03.047823 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76b6143-7bd0-4b59-ac5a-ad2795b26b66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:03.048999 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76b6143-7bd0-4b59-ac5a-ad2795b26b66 None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:03.049281 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76b6143-7bd0-4b59-ac5a-ad2795b26b66 None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:03.049555 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76b6143-7bd0-4b59-ac5a-ad2795b26b66 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:03.070740 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76b6143-7bd0-4b59-ac5a-ad2795b26b66 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 10 11:46:03.083694 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application [None req-a76b6143-7bd0-4b59-ac5a-ad2795b26b66 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 10 11:46:03.083694 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:46:03.083694 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:46:03.083694 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:46:03.083694 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:46:03.083694 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:46:03.083694 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:46:03.083694 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:46:03.083694 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:46:03.083694 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:46:03.083694 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:46:03.083694 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:46:03.083694 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 10 11:46:03.083694 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 10 11:46:03.083694 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 10 11:46:03.083694 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 10 11:46:03.084615 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 10 11:46:03.084615 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 10 11:46:03.084615 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 10 11:46:03.084615 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 10 11:46:03.084615 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:46:03.084615 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:46:03.084615 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:46:03.084615 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:46:03.084615 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:46:03.084615 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:46:03.084615 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:46:03.084615 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application with Lock( Oct 10 11:46:03.084615 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:46:03.084615 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:46:03.084615 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:46:03.084615 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:46:03.085378 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:46:03.085378 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:46:03.085378 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:46:03.085378 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:46:03.085378 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 10 11:46:03.085378 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 10 11:46:03.085378 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 10 11:46:03.085378 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 10 11:46:03.085378 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 10 11:46:03.085378 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 10 11:46:03.085378 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 10 11:46:03.085378 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Oct 10 11:46:03.087438 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 116/259] 10.222.0.46 () {56 vars in 1096 bytes} [Mon Oct 10 11:46:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 320 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:46:03.093063 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76b6143-7bd0-4b59-ac5a-ad2795b26b66 None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:03.096764 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a55d69-3662-4efd-a26b-5f82b222ff0f None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:03.199592 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2013e54-0e20-4614-8d3d-3bbdbf51e547 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:03.200243 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2013e54-0e20-4614-8d3d-3bbdbf51e547 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:03.200517 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2013e54-0e20-4614-8d3d-3bbdbf51e547 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:03.200798 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2013e54-0e20-4614-8d3d-3bbdbf51e547 None admin] PATH_INFO: `/v3/roles` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:03.201505 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2013e54-0e20-4614-8d3d-3bbdbf51e547 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:46:03.201806 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2013e54-0e20-4614-8d3d-3bbdbf51e547 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:03.203294 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2013e54-0e20-4614-8d3d-3bbdbf51e547 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:03.228856 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 144/260] 10.222.0.46 () {56 vars in 1111 bytes} [Mon Oct 10 11:46:02 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 330 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 11:46:03.234756 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2013e54-0e20-4614-8d3d-3bbdbf51e547 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:03.279297 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76b6143-7bd0-4b59-ac5a-ad2795b26b66 None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:03.336572 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:03.361494 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09b2fe9-be46-442a-9768-a4dac22d117d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:03.362495 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09b2fe9-be46-442a-9768-a4dac22d117d None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:03.363142 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09b2fe9-be46-442a-9768-a4dac22d117d None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:03.363640 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09b2fe9-be46-442a-9768-a4dac22d117d None admin] PATH_INFO: `/v3/roles` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:03.364525 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09b2fe9-be46-442a-9768-a4dac22d117d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:46:03.365032 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09b2fe9-be46-442a-9768-a4dac22d117d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:03.366644 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09b2fe9-be46-442a-9768-a4dac22d117d None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:03.387084 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 117/261] 10.222.0.46 () {56 vars in 1099 bytes} [Mon Oct 10 11:46:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:46:03.392984 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09b2fe9-be46-442a-9768-a4dac22d117d None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:03.430775 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:03.488590 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2013e54-0e20-4614-8d3d-3bbdbf51e547 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:03.529901 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe230be-5671-4ced-b7eb-ac657a16e0fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:03.530884 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe230be-5671-4ced-b7eb-ac657a16e0fc None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:03.531381 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe230be-5671-4ced-b7eb-ac657a16e0fc None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:03.531761 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe230be-5671-4ced-b7eb-ac657a16e0fc None admin] PATH_INFO: `/v3/users/6c3317696b14428a8edb1085c6cfcc3d` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:03.533411 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe230be-5671-4ced-b7eb-ac657a16e0fc None admin] RBAC: Authorizing `identity:get_user(user_id=6c3317696b14428a8edb1085c6cfcc3d)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:03.534993 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe230be-5671-4ced-b7eb-ac657a16e0fc None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:03.538801 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 145/262] 10.222.0.46 () {56 vars in 1177 bytes} [Mon Oct 10 11:46:03 2022] GET /identity/v3/users/6c3317696b14428a8edb1085c6cfcc3d => generated 283 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:46:03.548090 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe230be-5671-4ced-b7eb-ac657a16e0fc None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:03.678619 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09b2fe9-be46-442a-9768-a4dac22d117d None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:03.685625 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe230be-5671-4ced-b7eb-ac657a16e0fc None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:03.726132 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cf62fb-e1a7-4934-aa9e-9bec1ff35cff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:03.726927 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cf62fb-e1a7-4934-aa9e-9bec1ff35cff None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:03.727321 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cf62fb-e1a7-4934-aa9e-9bec1ff35cff None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:03.727847 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cf62fb-e1a7-4934-aa9e-9bec1ff35cff None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:03.753735 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:03.776304 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846099e6-b250-43b5-ad1d-8e908b62d8c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:03.781003 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846099e6-b250-43b5-ad1d-8e908b62d8c8 None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:03.781003 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846099e6-b250-43b5-ad1d-8e908b62d8c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:03.781003 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846099e6-b250-43b5-ad1d-8e908b62d8c8 None admin] PATH_INFO: `/v3/users/6c3317696b14428a8edb1085c6cfcc3d` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:03.783356 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846099e6-b250-43b5-ad1d-8e908b62d8c8 None admin] RBAC: Authorizing `identity:get_user(user_id=6c3317696b14428a8edb1085c6cfcc3d)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:03.785324 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846099e6-b250-43b5-ad1d-8e908b62d8c8 None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:03.786622 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 118/263] 10.222.0.46 () {56 vars in 1177 bytes} [Mon Oct 10 11:46:03 2022] GET /identity/v3/users/6c3317696b14428a8edb1085c6cfcc3d => generated 283 bytes in 394 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 11:46:03.792268 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846099e6-b250-43b5-ad1d-8e908b62d8c8 None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:03.834764 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06cf62fb-e1a7-4934-aa9e-9bec1ff35cff None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:03.836463 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06cf62fb-e1a7-4934-aa9e-9bec1ff35cff None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:03.941260 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:04.043184 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 146/264] 10.222.0.46 () {58 vars in 1282 bytes} [Mon Oct 10 11:46:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 495 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Oct 10 11:46:04.044677 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cf62fb-e1a7-4934-aa9e-9bec1ff35cff None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:04.076784 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846099e6-b250-43b5-ad1d-8e908b62d8c8 None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:04.113579 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:04.167420 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94feb816-fe2a-4d59-8c2e-b6197cb295ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:04.168055 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94feb816-fe2a-4d59-8c2e-b6197cb295ee None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:04.168343 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94feb816-fe2a-4d59-8c2e-b6197cb295ee None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:04.168674 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94feb816-fe2a-4d59-8c2e-b6197cb295ee None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:04.178516 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cf62fb-e1a7-4934-aa9e-9bec1ff35cff None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:04.213685 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963b4d60-09e2-440d-bc1e-f8fb43095034 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:04.214600 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963b4d60-09e2-440d-bc1e-f8fb43095034 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:04.215047 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963b4d60-09e2-440d-bc1e-f8fb43095034 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:04.215467 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963b4d60-09e2-440d-bc1e-f8fb43095034 None admin] PATH_INFO: `/v3/projects/55927449396d4c4f8470544dd7a23ed9` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:04.216746 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963b4d60-09e2-440d-bc1e-f8fb43095034 None admin] RBAC: Authorizing `identity:get_project(project_id=55927449396d4c4f8470544dd7a23ed9)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:04.218329 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963b4d60-09e2-440d-bc1e-f8fb43095034 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:04.220811 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 147/265] 10.222.0.46 () {56 vars in 1186 bytes} [Mon Oct 10 11:46:04 2022] GET /identity/v3/projects/55927449396d4c4f8470544dd7a23ed9 => generated 302 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:46:04.226650 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963b4d60-09e2-440d-bc1e-f8fb43095034 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:04.315583 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94feb816-fe2a-4d59-8c2e-b6197cb295ee None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:04.316988 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94feb816-fe2a-4d59-8c2e-b6197cb295ee None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:04.343704 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963b4d60-09e2-440d-bc1e-f8fb43095034 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:04.359311 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:04.384936 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c873619-557b-448c-b871-c4e23b3ed59f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:04.385940 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c873619-557b-448c-b871-c4e23b3ed59f None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:04.386442 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c873619-557b-448c-b871-c4e23b3ed59f None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:04.386989 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c873619-557b-448c-b871-c4e23b3ed59f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:04.387977 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c873619-557b-448c-b871-c4e23b3ed59f None admin] RBAC: Adding query filter params (role.id=6762beebc2bf4e6f804cbbe55a475e70, scope.project.id=55927449396d4c4f8470544dd7a23ed9, user.id=6c3317696b14428a8edb1085c6cfcc3d) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:46:04.388521 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c873619-557b-448c-b871-c4e23b3ed59f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:04.390306 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c873619-557b-448c-b871-c4e23b3ed59f None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:04.398693 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 148/266] 10.222.0.46 () {56 vars in 1374 bytes} [Mon Oct 10 11:46:04 2022] GET /identity/v3/role_assignments?user.id=6c3317696b14428a8edb1085c6cfcc3d&scope.project.id=55927449396d4c4f8470544dd7a23ed9&role.id=6762beebc2bf4e6f804cbbe55a475e70 => generated 608 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 11:46:04.443149 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:04.507784 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 119/267] 10.222.0.46 () {58 vars in 1282 bytes} [Mon Oct 10 11:46:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 716 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 10 11:46:04.513890 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c873619-557b-448c-b871-c4e23b3ed59f None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:04.572358 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80b22d9-d73a-42bc-8809-c8761aebb8a2 None None] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:04.572855 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80b22d9-d73a-42bc-8809-c8761aebb8a2 None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:04.573290 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80b22d9-d73a-42bc-8809-c8761aebb8a2 None None] PATH_INFO: `/` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:04.575179 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 120/268] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 10 11:46:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 10 11:46:04.581683 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed435710-32df-4e73-9ba0-ebc4f9aba86d None None] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:04.582189 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed435710-32df-4e73-9ba0-ebc4f9aba86d None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:04.582596 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed435710-32df-4e73-9ba0-ebc4f9aba86d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:04.615781 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.auth.core [None req-ed435710-32df-4e73-9ba0-ebc4f9aba86d None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:46:04.617222 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-ed435710-32df-4e73-9ba0-ebc4f9aba86d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:46:04.732737 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 121/269] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 10 11:46:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 10 11:46:04.734302 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1703efc9-a0e5-4e8f-98d3-d782c73978d5 None None] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:04.734654 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1703efc9-a0e5-4e8f-98d3-d782c73978d5 None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:04.735066 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1703efc9-a0e5-4e8f-98d3-d782c73978d5 None None] PATH_INFO: `/` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:04.736881 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 122/270] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 10 11:46:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:46:04.744333 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbb9067-a694-4bb3-8372-7c4e03403067 None None] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:04.744623 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbb9067-a694-4bb3-8372-7c4e03403067 None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:04.744898 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbb9067-a694-4bb3-8372-7c4e03403067 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:04.752380 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c873619-557b-448c-b871-c4e23b3ed59f None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:04.764211 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.auth.core [None req-cfbb9067-a694-4bb3-8372-7c4e03403067 None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:46:04.765420 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-cfbb9067-a694-4bb3-8372-7c4e03403067 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:46:04.792654 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0d55a6-1666-4d47-a213-7cd78a90c407 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:04.793899 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0d55a6-1666-4d47-a213-7cd78a90c407 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:04.794382 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0d55a6-1666-4d47-a213-7cd78a90c407 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:04.794984 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0d55a6-1666-4d47-a213-7cd78a90c407 None admin] PATH_INFO: `/v3/projects/55927449396d4c4f8470544dd7a23ed9` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:04.796400 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0d55a6-1666-4d47-a213-7cd78a90c407 None admin] RBAC: Authorizing `identity:get_project(project_id=55927449396d4c4f8470544dd7a23ed9)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:04.798399 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0d55a6-1666-4d47-a213-7cd78a90c407 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:04.800297 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 149/271] 10.222.0.46 () {56 vars in 1186 bytes} [Mon Oct 10 11:46:04 2022] GET /identity/v3/projects/55927449396d4c4f8470544dd7a23ed9 => generated 302 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:46:04.807474 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0d55a6-1666-4d47-a213-7cd78a90c407 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:04.854627 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 123/272] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 10 11:46:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 10 11:46:04.873143 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:04.945480 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0d55a6-1666-4d47-a213-7cd78a90c407 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:04.985580 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d82dd0-0c93-446d-95cd-db2e46250fb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:04.986566 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d82dd0-0c93-446d-95cd-db2e46250fb4 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:04.987096 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d82dd0-0c93-446d-95cd-db2e46250fb4 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:04.987596 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d82dd0-0c93-446d-95cd-db2e46250fb4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:04.988535 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d82dd0-0c93-446d-95cd-db2e46250fb4 None admin] RBAC: Adding query filter params (role.id=0d822a0674b6495fac623877a3b38025, scope.project.id=55927449396d4c4f8470544dd7a23ed9, user.id=6c3317696b14428a8edb1085c6cfcc3d) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:46:04.989047 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d82dd0-0c93-446d-95cd-db2e46250fb4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:04.990713 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d82dd0-0c93-446d-95cd-db2e46250fb4 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:05.000410 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 150/273] 10.222.0.46 () {56 vars in 1374 bytes} [Mon Oct 10 11:46:04 2022] GET /identity/v3/role_assignments?user.id=6c3317696b14428a8edb1085c6cfcc3d&scope.project.id=55927449396d4c4f8470544dd7a23ed9&role.id=0d822a0674b6495fac623877a3b38025 => generated 608 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:46:05.173561 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbb9067-a694-4bb3-8372-7c4e03403067 None None] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:05.188978 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f28f46-e059-4ed3-ab96-a91849650d38 None None] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:05.188978 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f28f46-e059-4ed3-ab96-a91849650d38 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:05.188978 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f28f46-e059-4ed3-ab96-a91849650d38 None None] PATH_INFO: `/` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:05.188978 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 151/274] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 10 11:46:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:46:05.195453 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:05.195824 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edf5b8b-441e-42a0-ae09-d817d2647bc9 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:05.196299 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edf5b8b-441e-42a0-ae09-d817d2647bc9 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:05.196764 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edf5b8b-441e-42a0-ae09-d817d2647bc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:05.217293 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-1edf5b8b-441e-42a0-ae09-d817d2647bc9 None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:46:05.218956 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-1edf5b8b-441e-42a0-ae09-d817d2647bc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:46:05.334641 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 152/275] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 10 11:46:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 10 11:46:05.335252 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edf5b8b-441e-42a0-ae09-d817d2647bc9 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:05.339799 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbb9067-a694-4bb3-8372-7c4e03403067 None None] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:05.381676 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:05.388801 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490d1eff-c652-4066-9803-1d31d1cdaba8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:05.389769 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490d1eff-c652-4066-9803-1d31d1cdaba8 None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:05.390219 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490d1eff-c652-4066-9803-1d31d1cdaba8 None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:05.390695 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490d1eff-c652-4066-9803-1d31d1cdaba8 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:05.404584 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490d1eff-c652-4066-9803-1d31d1cdaba8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 10 11:46:05.411073 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application [None req-490d1eff-c652-4066-9803-1d31d1cdaba8 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 10 11:46:05.411073 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:46:05.411073 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:46:05.411073 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:46:05.411073 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:46:05.411073 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:46:05.411073 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:46:05.411073 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:46:05.411073 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:46:05.411073 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:46:05.411073 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:46:05.411073 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:46:05.411073 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 10 11:46:05.411073 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 10 11:46:05.411073 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 10 11:46:05.411073 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 10 11:46:05.411971 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 10 11:46:05.411971 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 10 11:46:05.411971 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 10 11:46:05.411971 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 10 11:46:05.411971 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:46:05.411971 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:46:05.411971 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:46:05.411971 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:46:05.411971 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:46:05.411971 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:46:05.411971 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:46:05.411971 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application with Lock( Oct 10 11:46:05.411971 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:46:05.411971 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:46:05.411971 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:46:05.411971 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:46:05.412740 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:46:05.412740 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:46:05.412740 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:46:05.412740 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:46:05.412740 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 10 11:46:05.412740 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 10 11:46:05.412740 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 10 11:46:05.412740 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 10 11:46:05.412740 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 10 11:46:05.412740 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 10 11:46:05.412740 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 10 11:46:05.412740 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Oct 10 11:46:05.418700 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 124/276] 10.222.0.46 () {56 vars in 1099 bytes} [Mon Oct 10 11:46:05 2022] GET /identity/v3/roles/member => generated 84 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:46:05.424913 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490d1eff-c652-4066-9803-1d31d1cdaba8 None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:05.452546 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edf5b8b-441e-42a0-ae09-d817d2647bc9 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:05.498044 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4286525b-e703-4e49-8b60-1762173e5d0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:05.498699 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4286525b-e703-4e49-8b60-1762173e5d0e None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:05.498988 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4286525b-e703-4e49-8b60-1762173e5d0e None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:05.499255 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4286525b-e703-4e49-8b60-1762173e5d0e None admin] PATH_INFO: `/v3/domains/default` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:05.500364 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4286525b-e703-4e49-8b60-1762173e5d0e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:05.501875 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4286525b-e703-4e49-8b60-1762173e5d0e None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:05.503724 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 153/277] 10.222.0.46 () {56 vars in 1108 bytes} [Mon Oct 10 11:46:05 2022] GET /identity/v3/domains/default => generated 199 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:46:05.508831 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4286525b-e703-4e49-8b60-1762173e5d0e None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:05.575301 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:05.603138 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490d1eff-c652-4066-9803-1d31d1cdaba8 None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:05.615063 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4286525b-e703-4e49-8b60-1762173e5d0e None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:05.624810 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:05.643006 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2483c96-2c9c-495c-9af5-384e4cc841e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:05.643824 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2483c96-2c9c-495c-9af5-384e4cc841e0 None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:05.643930 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2483c96-2c9c-495c-9af5-384e4cc841e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:05.644163 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2483c96-2c9c-495c-9af5-384e4cc841e0 None admin] PATH_INFO: `/v3/roles` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:05.644841 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2483c96-2c9c-495c-9af5-384e4cc841e0 None admin] RBAC: Adding query filter params (name=member) {{(pid=81676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:46:05.645145 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2483c96-2c9c-495c-9af5-384e4cc841e0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:05.646559 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2483c96-2c9c-495c-9af5-384e4cc841e0 None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:05.651332 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038cb223-c45d-4b68-99b4-c9bbb90d2afa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:05.652404 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038cb223-c45d-4b68-99b4-c9bbb90d2afa None admin] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:05.652687 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038cb223-c45d-4b68-99b4-c9bbb90d2afa None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:05.652969 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038cb223-c45d-4b68-99b4-c9bbb90d2afa None admin] PATH_INFO: `/v3/users` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:05.653604 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038cb223-c45d-4b68-99b4-c9bbb90d2afa None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:05.655043 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038cb223-c45d-4b68-99b4-c9bbb90d2afa None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:05.659748 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 125/278] 10.222.0.46 () {56 vars in 1101 bytes} [Mon Oct 10 11:46:05 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 11:46:05.660569 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2483c96-2c9c-495c-9af5-384e4cc841e0 None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:05.738023 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 154/279] 10.222.0.46 () {60 vars in 1132 bytes} [Mon Oct 10 11:46:05 2022] POST /identity/v3/users => generated 293 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 11:46:05.738907 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038cb223-c45d-4b68-99b4-c9bbb90d2afa None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:05.791793 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2483c96-2c9c-495c-9af5-384e4cc841e0 None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:05.849972 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83752747-8d43-4fa7-8569-7380dbbea4f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:05.851827 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83752747-8d43-4fa7-8569-7380dbbea4f6 None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:05.852373 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83752747-8d43-4fa7-8569-7380dbbea4f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:05.852852 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83752747-8d43-4fa7-8569-7380dbbea4f6 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:05.857485 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:05.869816 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038cb223-c45d-4b68-99b4-c9bbb90d2afa None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:05.876613 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83752747-8d43-4fa7-8569-7380dbbea4f6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 10 11:46:05.880681 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application [None req-83752747-8d43-4fa7-8569-7380dbbea4f6 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 10 11:46:05.880681 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:46:05.880681 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:46:05.880681 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:46:05.880681 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:46:05.880681 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:46:05.880681 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:46:05.880681 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:46:05.880681 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:46:05.880681 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:46:05.880681 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:46:05.880681 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:46:05.880681 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 10 11:46:05.880681 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 10 11:46:05.880681 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 10 11:46:05.880681 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 10 11:46:05.881599 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 10 11:46:05.881599 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 10 11:46:05.881599 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 10 11:46:05.881599 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 10 11:46:05.881599 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:46:05.881599 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:46:05.881599 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:46:05.881599 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:46:05.881599 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:46:05.881599 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:46:05.881599 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:46:05.881599 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application with Lock( Oct 10 11:46:05.881599 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:46:05.881599 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:46:05.881599 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:46:05.881599 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:46:05.882426 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:46:05.882426 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:46:05.882426 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:46:05.882426 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:46:05.882426 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 10 11:46:05.882426 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 10 11:46:05.882426 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 10 11:46:05.882426 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 10 11:46:05.882426 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 10 11:46:05.882426 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 10 11:46:05.882426 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 10 11:46:05.882426 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Oct 10 11:46:05.889265 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 126/280] 10.222.0.46 () {56 vars in 1099 bytes} [Mon Oct 10 11:46:05 2022] GET /identity/v3/roles/reader => generated 84 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:46:05.890053 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83752747-8d43-4fa7-8569-7380dbbea4f6 None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:05.919951 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b5b02e-b323-46cc-a75c-a14694c9a00a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:05.920922 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b5b02e-b323-46cc-a75c-a14694c9a00a None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:05.921276 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b5b02e-b323-46cc-a75c-a14694c9a00a None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:05.921575 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b5b02e-b323-46cc-a75c-a14694c9a00a None admin] PATH_INFO: `/v3/users/c5698b4ba6e9482bb7731f8d4b9ef535` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:05.937589 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b5b02e-b323-46cc-a75c-a14694c9a00a None admin] RBAC: Authorizing `identity:get_user(user_id=c5698b4ba6e9482bb7731f8d4b9ef535)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:05.939699 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b5b02e-b323-46cc-a75c-a14694c9a00a None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:05.941464 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 155/281] 10.222.0.46 () {56 vars in 1177 bytes} [Mon Oct 10 11:46:05 2022] GET /identity/v3/users/c5698b4ba6e9482bb7731f8d4b9ef535 => generated 293 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:46:05.942764 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5da314-2f72-42ca-ae15-8f100f0590d0 None None] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:05.943046 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5da314-2f72-42ca-ae15-8f100f0590d0 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:05.943317 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5da314-2f72-42ca-ae15-8f100f0590d0 None None] PATH_INFO: `/` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:05.944841 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 156/282] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 10 11:46:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:46:05.949887 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5da314-2f72-42ca-ae15-8f100f0590d0 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:05.980929 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:06.040424 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83752747-8d43-4fa7-8569-7380dbbea4f6 None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:06.094558 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5da314-2f72-42ca-ae15-8f100f0590d0 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:06.096551 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cfe4940-3db9-49a0-bf96-94ba7bacd9f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:06.097184 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfe4940-3db9-49a0-bf96-94ba7bacd9f0 None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:06.097459 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfe4940-3db9-49a0-bf96-94ba7bacd9f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:06.097732 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfe4940-3db9-49a0-bf96-94ba7bacd9f0 None admin] PATH_INFO: `/v3/roles` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:06.098427 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cfe4940-3db9-49a0-bf96-94ba7bacd9f0 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:46:06.098824 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cfe4940-3db9-49a0-bf96-94ba7bacd9f0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:06.100219 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cfe4940-3db9-49a0-bf96-94ba7bacd9f0 None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:06.109742 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 127/283] 10.222.0.46 () {56 vars in 1101 bytes} [Mon Oct 10 11:46:05 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 11:46:06.115075 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b159879-7619-4246-baaa-5b3d309719c8 None None] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:06.115681 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b159879-7619-4246-baaa-5b3d309719c8 None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:06.116431 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b159879-7619-4246-baaa-5b3d309719c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:06.144296 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e151b9-99d3-4682-ad16-622361a2ed5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:06.144927 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e151b9-99d3-4682-ad16-622361a2ed5c None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:06.145210 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e151b9-99d3-4682-ad16-622361a2ed5c None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:06.145488 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e151b9-99d3-4682-ad16-622361a2ed5c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:06.146303 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e151b9-99d3-4682-ad16-622361a2ed5c None admin] RBAC: Adding query filter params (role.id=b2cb2a6e818e47338ce071e6fedfc673, user.id=c5698b4ba6e9482bb7731f8d4b9ef535, scope.system=all) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:46:06.146620 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e151b9-99d3-4682-ad16-622361a2ed5c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:06.148099 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e151b9-99d3-4682-ad16-622361a2ed5c None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:06.148999 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.auth.core [None req-6b159879-7619-4246-baaa-5b3d309719c8 None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:46:06.150416 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-6b159879-7619-4246-baaa-5b3d309719c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:46:06.154060 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 157/284] 10.222.0.46 () {56 vars in 1308 bytes} [Mon Oct 10 11:46:05 2022] GET /identity/v3/role_assignments?user.id=c5698b4ba6e9482bb7731f8d4b9ef535&scope.system=all&role.id=b2cb2a6e818e47338ce071e6fedfc673 => generated 226 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:46:06.154818 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e151b9-99d3-4682-ad16-622361a2ed5c None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:06.239800 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:06.255421 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 128/285] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 10 11:46:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 10 11:46:06.325372 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:06.357368 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e151b9-99d3-4682-ad16-622361a2ed5c None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:06.419124 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b4f63e-70c2-41b7-851f-9bb9a220d02e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:06.420580 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b4f63e-70c2-41b7-851f-9bb9a220d02e None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:06.421291 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b4f63e-70c2-41b7-851f-9bb9a220d02e None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:06.421997 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b4f63e-70c2-41b7-851f-9bb9a220d02e None admin] PATH_INFO: `/v3/users/d2da0904dbe64369af873368dbed2810` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:06.424390 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b4f63e-70c2-41b7-851f-9bb9a220d02e None admin] RBAC: Authorizing `identity:get_user(user_id=d2da0904dbe64369af873368dbed2810)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:06.427324 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b4f63e-70c2-41b7-851f-9bb9a220d02e None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:06.430503 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 158/286] 10.222.0.46 () {56 vars in 1177 bytes} [Mon Oct 10 11:46:06 2022] GET /identity/v3/users/d2da0904dbe64369af873368dbed2810 => generated 297 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:46:06.440316 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b159879-7619-4246-baaa-5b3d309719c8 None None] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:06.575276 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b4f63e-70c2-41b7-851f-9bb9a220d02e None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:06.616700 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b159879-7619-4246-baaa-5b3d309719c8 None None] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:06.694627 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0bbb9b-f176-4511-8a4d-85cc00052fce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:06.695303 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0bbb9b-f176-4511-8a4d-85cc00052fce None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:06.695585 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0bbb9b-f176-4511-8a4d-85cc00052fce None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:06.695856 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0bbb9b-f176-4511-8a4d-85cc00052fce None admin] PATH_INFO: `/v3/projects/55927449396d4c4f8470544dd7a23ed9` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:06.697103 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0bbb9b-f176-4511-8a4d-85cc00052fce None admin] RBAC: Authorizing `identity:get_project(project_id=55927449396d4c4f8470544dd7a23ed9)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:06.698552 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0bbb9b-f176-4511-8a4d-85cc00052fce None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:06.705208 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 129/287] 10.222.0.46 () {56 vars in 1186 bytes} [Mon Oct 10 11:46:06 2022] GET /identity/v3/projects/55927449396d4c4f8470544dd7a23ed9 => generated 302 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 11:46:06.710239 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0bbb9b-f176-4511-8a4d-85cc00052fce None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:06.731606 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b4f63e-70c2-41b7-851f-9bb9a220d02e None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:06.808386 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c43186-048d-43ce-9e34-2df48914abd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:06.809052 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c43186-048d-43ce-9e34-2df48914abd0 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:06.809337 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c43186-048d-43ce-9e34-2df48914abd0 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:06.809614 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c43186-048d-43ce-9e34-2df48914abd0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:06.814539 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:06.834284 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c43186-048d-43ce-9e34-2df48914abd0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 10 11:46:06.842723 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application [None req-80c43186-048d-43ce-9e34-2df48914abd0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 10 11:46:06.842723 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:46:06.842723 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:46:06.842723 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:46:06.842723 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:46:06.842723 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:46:06.842723 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:46:06.842723 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:46:06.842723 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:46:06.842723 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:46:06.842723 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:46:06.842723 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:46:06.842723 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 10 11:46:06.842723 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 10 11:46:06.842723 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 10 11:46:06.842723 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 10 11:46:06.843620 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 10 11:46:06.843620 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 10 11:46:06.843620 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 10 11:46:06.843620 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 10 11:46:06.843620 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:46:06.843620 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:46:06.843620 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:46:06.843620 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:46:06.843620 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:46:06.843620 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:46:06.843620 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:46:06.843620 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application with Lock( Oct 10 11:46:06.843620 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:46:06.843620 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:46:06.843620 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:46:06.843620 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:46:06.849519 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:46:06.849519 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:46:06.849519 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:46:06.849519 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:46:06.849519 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 10 11:46:06.849519 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 10 11:46:06.849519 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 10 11:46:06.849519 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 10 11:46:06.849519 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 10 11:46:06.849519 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 10 11:46:06.849519 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 10 11:46:06.849519 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Oct 10 11:46:06.849519 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 159/288] 10.222.0.46 () {56 vars in 1099 bytes} [Mon Oct 10 11:46:06 2022] GET /identity/v3/roles/member => generated 84 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 10 11:46:06.859801 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c43186-048d-43ce-9e34-2df48914abd0 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:06.886757 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0bbb9b-f176-4511-8a4d-85cc00052fce None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:06.892012 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:06.938816 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ecff26-7bfa-4046-a059-5e98e6342ef0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:06.940054 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ecff26-7bfa-4046-a059-5e98e6342ef0 None admin] REQUEST_METHOD: `PUT` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:06.940338 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ecff26-7bfa-4046-a059-5e98e6342ef0 None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:06.940635 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ecff26-7bfa-4046-a059-5e98e6342ef0 None admin] PATH_INFO: `/v3/projects/55927449396d4c4f8470544dd7a23ed9/users/d2da0904dbe64369af873368dbed2810/roles/54c5cfe84bab471db7992eee92787387` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:06.942849 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ecff26-7bfa-4046-a059-5e98e6342ef0 None admin] RBAC: Authorizing `identity:create_grant(project_id=55927449396d4c4f8470544dd7a23ed9, user_id=d2da0904dbe64369af873368dbed2810, role_id=54c5cfe84bab471db7992eee92787387)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:06.945321 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ecff26-7bfa-4046-a059-5e98e6342ef0 None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:07.021304 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c43186-048d-43ce-9e34-2df48914abd0 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:07.035249 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 130/289] 10.222.0.46 () {58 vars in 1439 bytes} [Mon Oct 10 11:46:06 2022] PUT /identity/v3/projects/55927449396d4c4f8470544dd7a23ed9/users/d2da0904dbe64369af873368dbed2810/roles/54c5cfe84bab471db7992eee92787387 => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 11:46:07.068922 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41da40e-5ec8-484d-9f46-6b6f4d6100ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:07.069573 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41da40e-5ec8-484d-9f46-6b6f4d6100ed None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:07.069854 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41da40e-5ec8-484d-9f46-6b6f4d6100ed None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:07.070132 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41da40e-5ec8-484d-9f46-6b6f4d6100ed None admin] PATH_INFO: `/v3/roles` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:07.072641 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41da40e-5ec8-484d-9f46-6b6f4d6100ed None admin] RBAC: Adding query filter params (name=member) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:46:07.072641 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41da40e-5ec8-484d-9f46-6b6f4d6100ed None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:07.072848 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41da40e-5ec8-484d-9f46-6b6f4d6100ed None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:07.083529 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 160/290] 10.222.0.46 () {56 vars in 1101 bytes} [Mon Oct 10 11:46:06 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:46:07.091387 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ecff26-7bfa-4046-a059-5e98e6342ef0 None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:07.237088 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ecff26-7bfa-4046-a059-5e98e6342ef0 None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:07.256711 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:07.276036 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c097d009-a441-427b-8f59-dd8a83be8eff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:07.276824 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c097d009-a441-427b-8f59-dd8a83be8eff None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:07.277085 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c097d009-a441-427b-8f59-dd8a83be8eff None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:07.277364 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c097d009-a441-427b-8f59-dd8a83be8eff None admin] PATH_INFO: `/v3/users/3d151593b3ff4f6aa2d2009f3b4aba81` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:07.278651 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c097d009-a441-427b-8f59-dd8a83be8eff None admin] RBAC: Authorizing `identity:get_user(user_id=3d151593b3ff4f6aa2d2009f3b4aba81)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:07.280162 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c097d009-a441-427b-8f59-dd8a83be8eff None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:07.281838 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 131/291] 10.222.0.46 () {56 vars in 1177 bytes} [Mon Oct 10 11:46:07 2022] GET /identity/v3/users/3d151593b3ff4f6aa2d2009f3b4aba81 => generated 297 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:46:07.287593 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41da40e-5ec8-484d-9f46-6b6f4d6100ed None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:07.397092 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:07.421433 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41da40e-5ec8-484d-9f46-6b6f4d6100ed None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:07.460557 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327b1d53-a465-4fef-87de-0b6a7298bf7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:07.461185 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327b1d53-a465-4fef-87de-0b6a7298bf7f None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:07.461470 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327b1d53-a465-4fef-87de-0b6a7298bf7f None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:07.461750 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327b1d53-a465-4fef-87de-0b6a7298bf7f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:07.536879 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327b1d53-a465-4fef-87de-0b6a7298bf7f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:07.538278 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327b1d53-a465-4fef-87de-0b6a7298bf7f None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:07.618109 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:07.646850 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 161/292] 10.222.0.46 () {58 vars in 1282 bytes} [Mon Oct 10 11:46:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 10 11:46:07.649071 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c097d009-a441-427b-8f59-dd8a83be8eff None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:07.777241 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c097d009-a441-427b-8f59-dd8a83be8eff None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:07.814617 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf464bd-ce38-4941-ae31-dd2c5427ccdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:07.815302 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf464bd-ce38-4941-ae31-dd2c5427ccdd None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:07.815628 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf464bd-ce38-4941-ae31-dd2c5427ccdd None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:07.815726 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf464bd-ce38-4941-ae31-dd2c5427ccdd None admin] PATH_INFO: `/v3/projects/55927449396d4c4f8470544dd7a23ed9` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:07.816968 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf464bd-ce38-4941-ae31-dd2c5427ccdd None admin] RBAC: Authorizing `identity:get_project(project_id=55927449396d4c4f8470544dd7a23ed9)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:07.818549 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf464bd-ce38-4941-ae31-dd2c5427ccdd None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:07.825303 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 132/293] 10.222.0.46 () {56 vars in 1186 bytes} [Mon Oct 10 11:46:07 2022] GET /identity/v3/projects/55927449396d4c4f8470544dd7a23ed9 => generated 302 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:46:07.826842 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327b1d53-a465-4fef-87de-0b6a7298bf7f None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:07.934884 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327b1d53-a465-4fef-87de-0b6a7298bf7f None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:07.972707 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da6b790-f159-4795-b765-e102fad29688 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:07.973327 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da6b790-f159-4795-b765-e102fad29688 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:07.973612 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da6b790-f159-4795-b765-e102fad29688 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:07.973889 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da6b790-f159-4795-b765-e102fad29688 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:07.974655 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da6b790-f159-4795-b765-e102fad29688 None admin] RBAC: Adding query filter params (role.id=b2cb2a6e818e47338ce071e6fedfc673, scope.project.id=55927449396d4c4f8470544dd7a23ed9, user.id=3d151593b3ff4f6aa2d2009f3b4aba81) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:46:07.974995 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da6b790-f159-4795-b765-e102fad29688 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:07.976443 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da6b790-f159-4795-b765-e102fad29688 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:07.985715 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 162/294] 10.222.0.46 () {56 vars in 1374 bytes} [Mon Oct 10 11:46:07 2022] GET /identity/v3/role_assignments?user.id=3d151593b3ff4f6aa2d2009f3b4aba81&scope.project.id=55927449396d4c4f8470544dd7a23ed9&role.id=b2cb2a6e818e47338ce071e6fedfc673 => generated 608 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:46:08.602633 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1f3769-d810-40f1-a2c9-f17e4369d4de None None] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:08.603163 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1f3769-d810-40f1-a2c9-f17e4369d4de None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:08.603638 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1f3769-d810-40f1-a2c9-f17e4369d4de None None] PATH_INFO: `/` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:08.606562 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 133/295] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 10 11:46:08 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:46:08.616952 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6ae2ca-2a59-4669-92f9-d1143adaea68 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:08.617448 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6ae2ca-2a59-4669-92f9-d1143adaea68 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:08.617521 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6ae2ca-2a59-4669-92f9-d1143adaea68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:08.639830 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-ec6ae2ca-2a59-4669-92f9-d1143adaea68 None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:46:08.641201 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-ec6ae2ca-2a59-4669-92f9-d1143adaea68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:46:08.690743 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:08.726609 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 163/296] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 10 11:46:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 10 11:46:08.763477 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19179af0-8cab-47d9-bfa3-2e3dd3560f06 None None] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:08.763704 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19179af0-8cab-47d9-bfa3-2e3dd3560f06 None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:08.764023 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19179af0-8cab-47d9-bfa3-2e3dd3560f06 None None] PATH_INFO: `/` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:08.765290 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 134/297] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 10 11:46:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:46:08.770849 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9faeabac-d73f-4985-8ecf-983eb50f19fb None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:08.771162 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9faeabac-d73f-4985-8ecf-983eb50f19fb None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:08.771400 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9faeabac-d73f-4985-8ecf-983eb50f19fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:08.790075 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-9faeabac-d73f-4985-8ecf-983eb50f19fb None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:46:08.791290 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-9faeabac-d73f-4985-8ecf-983eb50f19fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:46:08.862704 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 164/298] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 10 11:46:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 10 11:46:09.025787 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19179af0-8cab-47d9-bfa3-2e3dd3560f06 None None] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:09.052932 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee3206c-333a-4f6e-88df-e5a1386eb49b None None] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:09.053255 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee3206c-333a-4f6e-88df-e5a1386eb49b None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:09.053509 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee3206c-333a-4f6e-88df-e5a1386eb49b None None] PATH_INFO: `/` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:09.055480 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 165/299] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 10 11:46:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 10 11:46:09.062722 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb43845-f68e-4365-b3f1-90ff537686f2 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:09.063044 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb43845-f68e-4365-b3f1-90ff537686f2 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:09.063480 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb43845-f68e-4365-b3f1-90ff537686f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:09.093658 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-0eb43845-f68e-4365-b3f1-90ff537686f2 None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:46:09.096357 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:09.096855 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-0eb43845-f68e-4365-b3f1-90ff537686f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:46:09.164084 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19179af0-8cab-47d9-bfa3-2e3dd3560f06 None None] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:09.191554 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 166/300] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 10 11:46:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 10 11:46:09.192477 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb43845-f68e-4365-b3f1-90ff537686f2 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:09.221332 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6cd7e9-9ca8-46f0-98a4-a42c794cfad3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:09.222614 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6cd7e9-9ca8-46f0-98a4-a42c794cfad3 None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:09.223127 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6cd7e9-9ca8-46f0-98a4-a42c794cfad3 None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:09.223931 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6cd7e9-9ca8-46f0-98a4-a42c794cfad3 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:09.231590 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:09.241460 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6cd7e9-9ca8-46f0-98a4-a42c794cfad3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 10 11:46:09.248747 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application [None req-da6cd7e9-9ca8-46f0-98a4-a42c794cfad3 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 10 11:46:09.248747 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:46:09.248747 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:46:09.248747 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:46:09.248747 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:46:09.248747 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:46:09.248747 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:46:09.248747 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:46:09.248747 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:46:09.248747 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:46:09.248747 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:46:09.248747 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:46:09.248747 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 10 11:46:09.248747 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 10 11:46:09.248747 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 10 11:46:09.248747 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 10 11:46:09.249824 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 10 11:46:09.249824 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 10 11:46:09.249824 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 10 11:46:09.249824 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 10 11:46:09.249824 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:46:09.249824 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:46:09.249824 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:46:09.249824 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:46:09.249824 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:46:09.249824 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:46:09.249824 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:46:09.249824 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application with Lock( Oct 10 11:46:09.249824 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:46:09.249824 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:46:09.249824 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:46:09.249824 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:46:09.250829 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:46:09.250829 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:46:09.250829 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:46:09.250829 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:46:09.250829 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 10 11:46:09.250829 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 10 11:46:09.250829 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 10 11:46:09.250829 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 10 11:46:09.250829 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 10 11:46:09.250829 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 10 11:46:09.250829 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 10 11:46:09.250829 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Oct 10 11:46:09.253489 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 135/301] 10.222.0.46 () {56 vars in 1099 bytes} [Mon Oct 10 11:46:09 2022] GET /identity/v3/roles/reader => generated 84 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:46:09.260612 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6cd7e9-9ca8-46f0-98a4-a42c794cfad3 None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:09.332912 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb43845-f68e-4365-b3f1-90ff537686f2 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:09.383387 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4258895-b732-4ef3-ab22-0f4cbc0aa0d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:09.384647 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4258895-b732-4ef3-ab22-0f4cbc0aa0d5 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:09.385436 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4258895-b732-4ef3-ab22-0f4cbc0aa0d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:09.386409 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4258895-b732-4ef3-ab22-0f4cbc0aa0d5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:09.388177 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4258895-b732-4ef3-ab22-0f4cbc0aa0d5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:09.390782 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4258895-b732-4ef3-ab22-0f4cbc0aa0d5 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:09.392398 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6cd7e9-9ca8-46f0-98a4-a42c794cfad3 None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:09.393261 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 167/302] 10.222.0.46 () {56 vars in 1108 bytes} [Mon Oct 10 11:46:09 2022] GET /identity/v3/domains/default => generated 199 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:46:09.398403 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4258895-b732-4ef3-ab22-0f4cbc0aa0d5 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:09.411599 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:09.435255 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb88b086-465b-4de8-a926-68411a232d7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:09.435869 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb88b086-465b-4de8-a926-68411a232d7a None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:09.436158 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb88b086-465b-4de8-a926-68411a232d7a None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:09.436438 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb88b086-465b-4de8-a926-68411a232d7a None admin] PATH_INFO: `/v3/roles` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:09.437237 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb88b086-465b-4de8-a926-68411a232d7a None admin] RBAC: Adding query filter params (name=reader) {{(pid=81676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:46:09.437483 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb88b086-465b-4de8-a926-68411a232d7a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:09.439120 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb88b086-465b-4de8-a926-68411a232d7a None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:09.455396 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 136/303] 10.222.0.46 () {56 vars in 1101 bytes} [Mon Oct 10 11:46:09 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:46:09.457353 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb88b086-465b-4de8-a926-68411a232d7a None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:09.546389 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:09.551408 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4258895-b732-4ef3-ab22-0f4cbc0aa0d5 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:09.591417 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb88b086-465b-4de8-a926-68411a232d7a None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:09.597196 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff85134-7463-458b-9246-8052620985e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:09.597854 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff85134-7463-458b-9246-8052620985e2 None admin] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:09.598166 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff85134-7463-458b-9246-8052620985e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:09.598448 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff85134-7463-458b-9246-8052620985e2 None admin] PATH_INFO: `/v3/groups` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:09.599250 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff85134-7463-458b-9246-8052620985e2 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:09.601102 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff85134-7463-458b-9246-8052620985e2 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:09.631509 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef744229-ed5a-40a3-819d-2adfec0acdc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:09.632230 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef744229-ed5a-40a3-819d-2adfec0acdc5 None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:09.632513 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef744229-ed5a-40a3-819d-2adfec0acdc5 None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:09.632789 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef744229-ed5a-40a3-819d-2adfec0acdc5 None admin] PATH_INFO: `/v3/users/ea84c3ded2a749c1855718a81537dab9` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:09.649908 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef744229-ed5a-40a3-819d-2adfec0acdc5 None admin] RBAC: Authorizing `identity:get_user(user_id=ea84c3ded2a749c1855718a81537dab9)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:09.651528 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef744229-ed5a-40a3-819d-2adfec0acdc5 None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:09.653436 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 137/304] 10.222.0.46 () {56 vars in 1177 bytes} [Mon Oct 10 11:46:09 2022] GET /identity/v3/users/ea84c3ded2a749c1855718a81537dab9 => generated 293 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:46:09.654126 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef744229-ed5a-40a3-819d-2adfec0acdc5 None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:09.674609 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 168/305] 10.222.0.46 () {60 vars in 1134 bytes} [Mon Oct 10 11:46:09 2022] POST /identity/v3/groups => generated 229 bytes in 277 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 10 11:46:09.675502 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff85134-7463-458b-9246-8052620985e2 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:09.705527 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:09.836491 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef744229-ed5a-40a3-819d-2adfec0acdc5 None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:09.848133 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff85134-7463-458b-9246-8052620985e2 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:09.882823 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd185f0b-9e7d-4a25-913b-fda09c0218de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:09.883874 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd185f0b-9e7d-4a25-913b-fda09c0218de None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:09.884309 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd185f0b-9e7d-4a25-913b-fda09c0218de None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:09.884762 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd185f0b-9e7d-4a25-913b-fda09c0218de None admin] PATH_INFO: `/v3/roles/member` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:09.897175 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd185f0b-9e7d-4a25-913b-fda09c0218de None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 10 11:46:09.902661 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application [None req-dd185f0b-9e7d-4a25-913b-fda09c0218de None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 10 11:46:09.902661 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:46:09.902661 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:46:09.902661 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:46:09.902661 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:46:09.902661 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:46:09.902661 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:46:09.902661 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:46:09.902661 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:46:09.902661 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:46:09.902661 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:46:09.902661 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:46:09.902661 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 10 11:46:09.902661 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 10 11:46:09.902661 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 10 11:46:09.902661 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 10 11:46:09.903729 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 10 11:46:09.903729 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 10 11:46:09.903729 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 10 11:46:09.903729 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 10 11:46:09.903729 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:46:09.903729 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:46:09.903729 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:46:09.903729 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:46:09.903729 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:46:09.903729 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:46:09.903729 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:46:09.903729 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application with Lock( Oct 10 11:46:09.903729 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:46:09.903729 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:46:09.903729 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:46:09.903729 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:46:09.904517 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:46:09.904517 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:46:09.904517 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:46:09.904517 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:46:09.904517 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 10 11:46:09.904517 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 10 11:46:09.904517 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 10 11:46:09.904517 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 10 11:46:09.904517 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 10 11:46:09.904517 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 10 11:46:09.904517 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 10 11:46:09.904517 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Oct 10 11:46:09.906639 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 138/306] 10.222.0.46 () {56 vars in 1099 bytes} [Mon Oct 10 11:46:09 2022] GET /identity/v3/roles/member => generated 84 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:46:09.908564 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549d7b88-d32e-4ceb-94b0-c38366340fa2 None None] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:09.908869 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549d7b88-d32e-4ceb-94b0-c38366340fa2 None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:09.909285 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549d7b88-d32e-4ceb-94b0-c38366340fa2 None None] PATH_INFO: `/` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:09.911045 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 139/307] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 10 11:46:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:46:09.916238 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549d7b88-d32e-4ceb-94b0-c38366340fa2 None None] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:09.919091 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5bf70b3-4440-4278-8e29-c9b8e975e93d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:09.919756 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bf70b3-4440-4278-8e29-c9b8e975e93d None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:09.920178 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bf70b3-4440-4278-8e29-c9b8e975e93d None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:09.920457 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bf70b3-4440-4278-8e29-c9b8e975e93d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:09.921238 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5bf70b3-4440-4278-8e29-c9b8e975e93d None admin] RBAC: Adding query filter params (role.id=54c5cfe84bab471db7992eee92787387, user.id=ea84c3ded2a749c1855718a81537dab9, scope.system=all) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:46:09.921628 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5bf70b3-4440-4278-8e29-c9b8e975e93d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:09.923861 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5bf70b3-4440-4278-8e29-c9b8e975e93d None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:09.930262 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 169/308] 10.222.0.46 () {56 vars in 1308 bytes} [Mon Oct 10 11:46:09 2022] GET /identity/v3/role_assignments?user.id=ea84c3ded2a749c1855718a81537dab9&scope.system=all&role.id=54c5cfe84bab471db7992eee92787387 => generated 226 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:46:09.931522 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e02665f-2fda-44e6-af7b-a9bf79ad3f88 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:09.931810 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e02665f-2fda-44e6-af7b-a9bf79ad3f88 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:09.932084 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e02665f-2fda-44e6-af7b-a9bf79ad3f88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:09.954909 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-3e02665f-2fda-44e6-af7b-a9bf79ad3f88 None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:46:09.956252 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-3e02665f-2fda-44e6-af7b-a9bf79ad3f88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:46:09.988000 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:10.109537 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:10.112388 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 170/309] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 10 11:46:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 10 11:46:10.129239 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549d7b88-d32e-4ceb-94b0-c38366340fa2 None None] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:10.197046 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6caccc85-fac4-4c32-8fbb-355ffd326c64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:10.198165 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6caccc85-fac4-4c32-8fbb-355ffd326c64 None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:10.198659 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6caccc85-fac4-4c32-8fbb-355ffd326c64 None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:10.199172 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6caccc85-fac4-4c32-8fbb-355ffd326c64 None admin] PATH_INFO: `/v3/roles` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:10.200389 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6caccc85-fac4-4c32-8fbb-355ffd326c64 None admin] RBAC: Adding query filter params (name=member) {{(pid=81676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:46:10.200922 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6caccc85-fac4-4c32-8fbb-355ffd326c64 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:10.203506 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6caccc85-fac4-4c32-8fbb-355ffd326c64 None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:10.221430 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 140/310] 10.222.0.46 () {56 vars in 1101 bytes} [Mon Oct 10 11:46:09 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 11:46:10.229495 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e02665f-2fda-44e6-af7b-a9bf79ad3f88 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:10.389229 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e02665f-2fda-44e6-af7b-a9bf79ad3f88 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:10.453745 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a00045d-bebc-438e-8325-017de69cd797 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:10.455266 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a00045d-bebc-438e-8325-017de69cd797 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:10.455727 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a00045d-bebc-438e-8325-017de69cd797 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:10.456174 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a00045d-bebc-438e-8325-017de69cd797 None admin] PATH_INFO: `/v3/users/c5698b4ba6e9482bb7731f8d4b9ef535` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:10.457982 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a00045d-bebc-438e-8325-017de69cd797 None admin] RBAC: Authorizing `identity:get_user(user_id=c5698b4ba6e9482bb7731f8d4b9ef535)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:10.460375 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a00045d-bebc-438e-8325-017de69cd797 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:10.463322 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 171/311] 10.222.0.46 () {56 vars in 1177 bytes} [Mon Oct 10 11:46:10 2022] GET /identity/v3/users/c5698b4ba6e9482bb7731f8d4b9ef535 => generated 293 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:46:10.472193 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6caccc85-fac4-4c32-8fbb-355ffd326c64 None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:10.535830 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a00045d-bebc-438e-8325-017de69cd797 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:10.560741 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:10.638984 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6caccc85-fac4-4c32-8fbb-355ffd326c64 None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:10.681857 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e7b4a8-6be4-4495-911e-096befdc8c36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:10.682942 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e7b4a8-6be4-4495-911e-096befdc8c36 None admin] REQUEST_METHOD: `PUT` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:10.683470 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e7b4a8-6be4-4495-911e-096befdc8c36 None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:10.683925 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e7b4a8-6be4-4495-911e-096befdc8c36 None admin] PATH_INFO: `/v3/system/users/c5698b4ba6e9482bb7731f8d4b9ef535/roles/b2cb2a6e818e47338ce071e6fedfc673` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:10.685817 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e7b4a8-6be4-4495-911e-096befdc8c36 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=c5698b4ba6e9482bb7731f8d4b9ef535, role_id=b2cb2a6e818e47338ce071e6fedfc673)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:10.687460 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e7b4a8-6be4-4495-911e-096befdc8c36 None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:10.742493 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a00045d-bebc-438e-8325-017de69cd797 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:10.797007 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 141/312] 10.222.0.46 () {58 vars in 1334 bytes} [Mon Oct 10 11:46:10 2022] PUT /identity/v3/system/users/c5698b4ba6e9482bb7731f8d4b9ef535/roles/b2cb2a6e818e47338ce071e6fedfc673 => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 11:46:10.812370 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f963f76f-e5db-4b00-90bc-c6e23128a32b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:10.813449 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f963f76f-e5db-4b00-90bc-c6e23128a32b None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:10.813873 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f963f76f-e5db-4b00-90bc-c6e23128a32b None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:10.814412 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f963f76f-e5db-4b00-90bc-c6e23128a32b None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:10.834662 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f963f76f-e5db-4b00-90bc-c6e23128a32b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 10 11:46:10.843471 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application [None req-f963f76f-e5db-4b00-90bc-c6e23128a32b None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 10 11:46:10.843471 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:46:10.843471 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:46:10.843471 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:46:10.843471 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:46:10.843471 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:46:10.843471 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:46:10.843471 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:46:10.843471 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:46:10.843471 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:46:10.843471 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:46:10.843471 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:46:10.843471 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 10 11:46:10.843471 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 10 11:46:10.843471 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 10 11:46:10.843471 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 10 11:46:10.844987 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 10 11:46:10.844987 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 10 11:46:10.844987 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 10 11:46:10.844987 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 10 11:46:10.844987 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:46:10.844987 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:46:10.844987 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:46:10.844987 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:46:10.844987 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:46:10.844987 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:46:10.844987 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:46:10.844987 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application with Lock( Oct 10 11:46:10.844987 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:46:10.844987 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:46:10.844987 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:46:10.844987 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:46:10.845806 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:46:10.845806 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:46:10.845806 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:46:10.845806 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:46:10.845806 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 10 11:46:10.845806 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 10 11:46:10.845806 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 10 11:46:10.845806 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 10 11:46:10.845806 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 10 11:46:10.845806 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 10 11:46:10.845806 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 10 11:46:10.845806 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Oct 10 11:46:10.846720 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 172/313] 10.222.0.46 () {56 vars in 1099 bytes} [Mon Oct 10 11:46:10 2022] GET /identity/v3/roles/reader => generated 84 bytes in 311 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 10 11:46:10.856336 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e7b4a8-6be4-4495-911e-096befdc8c36 None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:10.884031 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:11.019994 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e7b4a8-6be4-4495-911e-096befdc8c36 None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:11.076140 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61efcdd7-e2b7-43d0-a446-60da2288b587 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:11.076762 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61efcdd7-e2b7-43d0-a446-60da2288b587 None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:11.077041 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61efcdd7-e2b7-43d0-a446-60da2288b587 None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:11.077315 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61efcdd7-e2b7-43d0-a446-60da2288b587 None admin] PATH_INFO: `/v3/roles` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:11.077989 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61efcdd7-e2b7-43d0-a446-60da2288b587 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:46:11.078294 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61efcdd7-e2b7-43d0-a446-60da2288b587 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:11.080333 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61efcdd7-e2b7-43d0-a446-60da2288b587 None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:11.090904 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 142/314] 10.222.0.46 () {56 vars in 1101 bytes} [Mon Oct 10 11:46:10 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:46:11.097457 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f963f76f-e5db-4b00-90bc-c6e23128a32b None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:11.186695 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:11.207815 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f963f76f-e5db-4b00-90bc-c6e23128a32b None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:11.245524 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291c50de-ba0a-469b-8544-288e65c89ac2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:11.246198 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291c50de-ba0a-469b-8544-288e65c89ac2 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:11.246986 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291c50de-ba0a-469b-8544-288e65c89ac2 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:11.247274 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291c50de-ba0a-469b-8544-288e65c89ac2 None admin] PATH_INFO: `/v3/users/d2da0904dbe64369af873368dbed2810` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:11.248902 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291c50de-ba0a-469b-8544-288e65c89ac2 None admin] RBAC: Authorizing `identity:get_user(user_id=d2da0904dbe64369af873368dbed2810)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:11.250510 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291c50de-ba0a-469b-8544-288e65c89ac2 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:11.252874 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 173/315] 10.222.0.46 () {56 vars in 1177 bytes} [Mon Oct 10 11:46:11 2022] GET /identity/v3/users/d2da0904dbe64369af873368dbed2810 => generated 297 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:46:11.259518 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61efcdd7-e2b7-43d0-a446-60da2288b587 None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:11.362392 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:11.398358 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61efcdd7-e2b7-43d0-a446-60da2288b587 None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:11.451790 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afdb0fa7-e2b6-4df0-9d4c-c37b7e17dd44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:11.452444 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdb0fa7-e2b6-4df0-9d4c-c37b7e17dd44 None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:11.452727 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdb0fa7-e2b6-4df0-9d4c-c37b7e17dd44 None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:11.453008 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdb0fa7-e2b6-4df0-9d4c-c37b7e17dd44 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:11.546962 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afdb0fa7-e2b6-4df0-9d4c-c37b7e17dd44 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:11.550248 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afdb0fa7-e2b6-4df0-9d4c-c37b7e17dd44 None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:11.703727 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:11.741512 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 143/316] 10.222.0.46 () {58 vars in 1282 bytes} [Mon Oct 10 11:46:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 482 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 10 11:46:11.748107 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291c50de-ba0a-469b-8544-288e65c89ac2 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:11.861789 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291c50de-ba0a-469b-8544-288e65c89ac2 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:11.898950 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c760aca-02cc-4d55-9ce5-11090cc89dd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:11.899597 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c760aca-02cc-4d55-9ce5-11090cc89dd6 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:11.899878 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c760aca-02cc-4d55-9ce5-11090cc89dd6 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:11.900155 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c760aca-02cc-4d55-9ce5-11090cc89dd6 None admin] PATH_INFO: `/v3/projects/55927449396d4c4f8470544dd7a23ed9` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:11.901333 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c760aca-02cc-4d55-9ce5-11090cc89dd6 None admin] RBAC: Authorizing `identity:get_project(project_id=55927449396d4c4f8470544dd7a23ed9)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:11.902827 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c760aca-02cc-4d55-9ce5-11090cc89dd6 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:11.904725 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 174/317] 10.222.0.46 () {56 vars in 1186 bytes} [Mon Oct 10 11:46:11 2022] GET /identity/v3/projects/55927449396d4c4f8470544dd7a23ed9 => generated 302 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:46:11.911186 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afdb0fa7-e2b6-4df0-9d4c-c37b7e17dd44 None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:12.030803 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afdb0fa7-e2b6-4df0-9d4c-c37b7e17dd44 None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:12.093687 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80928195-b23d-4bd2-b16c-e95fcc21b76a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:12.094813 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80928195-b23d-4bd2-b16c-e95fcc21b76a None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:12.095315 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80928195-b23d-4bd2-b16c-e95fcc21b76a None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:12.095801 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80928195-b23d-4bd2-b16c-e95fcc21b76a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:12.097159 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80928195-b23d-4bd2-b16c-e95fcc21b76a None admin] RBAC: Adding query filter params (role.id=54c5cfe84bab471db7992eee92787387, scope.project.id=55927449396d4c4f8470544dd7a23ed9, user.id=d2da0904dbe64369af873368dbed2810) {{(pid=81676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:46:12.097723 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80928195-b23d-4bd2-b16c-e95fcc21b76a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:12.100471 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80928195-b23d-4bd2-b16c-e95fcc21b76a None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:12.113013 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 144/318] 10.222.0.46 () {56 vars in 1374 bytes} [Mon Oct 10 11:46:11 2022] GET /identity/v3/role_assignments?user.id=d2da0904dbe64369af873368dbed2810&scope.project.id=55927449396d4c4f8470544dd7a23ed9&role.id=54c5cfe84bab471db7992eee92787387 => generated 608 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:46:12.707780 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7549f2-4e5d-451d-8947-79d3b8dcaec6 None None] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:12.708276 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7549f2-4e5d-451d-8947-79d3b8dcaec6 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:12.708748 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7549f2-4e5d-451d-8947-79d3b8dcaec6 None None] PATH_INFO: `/` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:12.711320 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 175/319] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 10 11:46:12 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:46:12.717471 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f680a07-0569-46ff-a7ca-08ebd22636c7 None None] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:12.718261 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f680a07-0569-46ff-a7ca-08ebd22636c7 None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:12.718261 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f680a07-0569-46ff-a7ca-08ebd22636c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:12.738575 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.auth.core [None req-0f680a07-0569-46ff-a7ca-08ebd22636c7 None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:46:12.739728 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-0f680a07-0569-46ff-a7ca-08ebd22636c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:46:12.791779 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:12.825384 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 145/320] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 10 11:46:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 10 11:46:12.993853 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea8c8ba-72e8-4398-9340-c76c1d93e26a None None] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:12.994586 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea8c8ba-72e8-4398-9340-c76c1d93e26a None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:12.994980 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea8c8ba-72e8-4398-9340-c76c1d93e26a None None] PATH_INFO: `/` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:12.997685 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 176/321] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 10 11:46:12 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:46:13.006890 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058e03e1-a436-48de-bc32-e8e4b341b45f None None] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:13.007394 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058e03e1-a436-48de-bc32-e8e4b341b45f None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:13.007881 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058e03e1-a436-48de-bc32-e8e4b341b45f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:13.040166 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.auth.core [None req-058e03e1-a436-48de-bc32-e8e4b341b45f None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:46:13.042150 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-058e03e1-a436-48de-bc32-e8e4b341b45f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:46:13.103683 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea8c8ba-72e8-4398-9340-c76c1d93e26a None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:13.131194 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:13.156437 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 146/322] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 10 11:46:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 10 11:46:13.246587 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96291a94-cfbe-4b9e-aa7d-31c6f6f40e1d None None] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:13.246918 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96291a94-cfbe-4b9e-aa7d-31c6f6f40e1d None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:13.247094 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96291a94-cfbe-4b9e-aa7d-31c6f6f40e1d None None] PATH_INFO: `/` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:13.248951 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 147/323] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 10 11:46:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 10 11:46:13.254554 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba8b6c4-26cf-4f0c-a0f1-250af6f606bf None None] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:13.254880 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba8b6c4-26cf-4f0c-a0f1-250af6f606bf None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:13.255144 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba8b6c4-26cf-4f0c-a0f1-250af6f606bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:13.274919 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.auth.core [None req-6ba8b6c4-26cf-4f0c-a0f1-250af6f606bf None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:46:13.276120 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-6ba8b6c4-26cf-4f0c-a0f1-250af6f606bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:46:13.307953 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea8c8ba-72e8-4398-9340-c76c1d93e26a None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:13.358890 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 148/324] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 10 11:46:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 10 11:46:13.362237 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1487fdc6-604f-497f-8397-3ba52f93b723 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:13.363063 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1487fdc6-604f-497f-8397-3ba52f93b723 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:13.363475 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1487fdc6-604f-497f-8397-3ba52f93b723 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:13.363871 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1487fdc6-604f-497f-8397-3ba52f93b723 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:13.376285 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1487fdc6-604f-497f-8397-3ba52f93b723 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 10 11:46:13.381529 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application [None req-1487fdc6-604f-497f-8397-3ba52f93b723 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 10 11:46:13.381529 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:46:13.381529 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:46:13.381529 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:46:13.381529 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:46:13.381529 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:46:13.381529 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:46:13.381529 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:46:13.381529 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:46:13.381529 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:46:13.381529 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:46:13.381529 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:46:13.381529 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 10 11:46:13.381529 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 10 11:46:13.381529 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 10 11:46:13.381529 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 10 11:46:13.383218 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 10 11:46:13.383218 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 10 11:46:13.383218 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 10 11:46:13.383218 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 10 11:46:13.383218 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:46:13.383218 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:46:13.383218 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:46:13.383218 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:46:13.383218 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:46:13.383218 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:46:13.383218 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:46:13.383218 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application with Lock( Oct 10 11:46:13.383218 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:46:13.383218 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:46:13.383218 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:46:13.383218 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:46:13.384306 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:46:13.384306 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:46:13.384306 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:46:13.384306 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:46:13.384306 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 10 11:46:13.384306 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 10 11:46:13.384306 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 10 11:46:13.384306 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 10 11:46:13.384306 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 10 11:46:13.384306 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 10 11:46:13.384306 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 10 11:46:13.384306 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Oct 10 11:46:13.384306 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 177/325] 10.222.0.46 () {56 vars in 1099 bytes} [Mon Oct 10 11:46:13 2022] GET /identity/v3/roles/reader => generated 84 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 10 11:46:13.389626 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba8b6c4-26cf-4f0c-a0f1-250af6f606bf None None] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:13.427995 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:13.516070 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba8b6c4-26cf-4f0c-a0f1-250af6f606bf None None] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:13.556622 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ee367b-74ee-4ae1-8f41-1ecd4d18dc23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:13.557250 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ee367b-74ee-4ae1-8f41-1ecd4d18dc23 None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:13.557528 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ee367b-74ee-4ae1-8f41-1ecd4d18dc23 None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:13.557807 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ee367b-74ee-4ae1-8f41-1ecd4d18dc23 None admin] PATH_INFO: `/v3/roles` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:13.558495 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ee367b-74ee-4ae1-8f41-1ecd4d18dc23 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:46:13.558837 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ee367b-74ee-4ae1-8f41-1ecd4d18dc23 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:13.560254 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ee367b-74ee-4ae1-8f41-1ecd4d18dc23 None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:13.570902 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 149/326] 10.222.0.46 () {56 vars in 1101 bytes} [Mon Oct 10 11:46:13 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 11:46:13.576622 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1487fdc6-604f-497f-8397-3ba52f93b723 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:13.660112 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ee367b-74ee-4ae1-8f41-1ecd4d18dc23 None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:13.678815 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:13.716384 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1487fdc6-604f-497f-8397-3ba52f93b723 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:13.766731 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37f533a-59d4-4235-b637-43d7ed65d981 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:13.767561 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37f533a-59d4-4235-b637-43d7ed65d981 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:13.768046 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37f533a-59d4-4235-b637-43d7ed65d981 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:13.768462 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37f533a-59d4-4235-b637-43d7ed65d981 None admin] PATH_INFO: `/v3/users/ea84c3ded2a749c1855718a81537dab9` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:13.770290 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37f533a-59d4-4235-b637-43d7ed65d981 None admin] RBAC: Authorizing `identity:get_user(user_id=ea84c3ded2a749c1855718a81537dab9)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:13.772899 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37f533a-59d4-4235-b637-43d7ed65d981 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:13.775875 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 178/327] 10.222.0.46 () {56 vars in 1177 bytes} [Mon Oct 10 11:46:13 2022] GET /identity/v3/users/ea84c3ded2a749c1855718a81537dab9 => generated 293 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 11:46:13.776879 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37f533a-59d4-4235-b637-43d7ed65d981 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:13.795370 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:13.802917 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ee367b-74ee-4ae1-8f41-1ecd4d18dc23 None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:13.845440 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8b2dfa-d1a1-4e48-808d-d3b7e17b929e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:13.846247 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8b2dfa-d1a1-4e48-808d-d3b7e17b929e None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:13.846616 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8b2dfa-d1a1-4e48-808d-d3b7e17b929e None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:13.847039 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8b2dfa-d1a1-4e48-808d-d3b7e17b929e None admin] PATH_INFO: `/v3/domains/default` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:13.848472 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8b2dfa-d1a1-4e48-808d-d3b7e17b929e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:13.850039 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8b2dfa-d1a1-4e48-808d-d3b7e17b929e None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:13.851817 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 150/328] 10.222.0.46 () {56 vars in 1108 bytes} [Mon Oct 10 11:46:13 2022] GET /identity/v3/domains/default => generated 199 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 11:46:13.859043 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8b2dfa-d1a1-4e48-808d-d3b7e17b929e None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:13.901489 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37f533a-59d4-4235-b637-43d7ed65d981 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:13.953713 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a6b376-1c17-4486-9492-eba72d4fc5b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:13.955066 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a6b376-1c17-4486-9492-eba72d4fc5b6 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:13.955513 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a6b376-1c17-4486-9492-eba72d4fc5b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:13.955955 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a6b376-1c17-4486-9492-eba72d4fc5b6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:13.972663 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a6b376-1c17-4486-9492-eba72d4fc5b6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 10 11:46:13.981493 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application [None req-c1a6b376-1c17-4486-9492-eba72d4fc5b6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 10 11:46:13.981493 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:46:13.981493 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:46:13.981493 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:46:13.981493 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:46:13.981493 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:46:13.981493 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:46:13.981493 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:46:13.981493 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:46:13.981493 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:46:13.981493 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:46:13.981493 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:46:13.981493 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 10 11:46:13.981493 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 10 11:46:13.981493 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 10 11:46:13.981493 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 10 11:46:13.983157 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 10 11:46:13.983157 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 10 11:46:13.983157 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 10 11:46:13.983157 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 10 11:46:13.983157 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:46:13.983157 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:46:13.983157 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:46:13.983157 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:46:13.983157 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:46:13.983157 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:46:13.983157 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:46:13.983157 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application with Lock( Oct 10 11:46:13.983157 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:46:13.983157 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:46:13.983157 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:46:13.983157 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:46:13.984673 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:46:13.984673 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:46:13.984673 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:46:13.984673 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:46:13.984673 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 10 11:46:13.984673 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 10 11:46:13.984673 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 10 11:46:13.984673 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 10 11:46:13.984673 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 10 11:46:13.984673 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 10 11:46:13.984673 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 10 11:46:13.984673 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Oct 10 11:46:13.984673 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 179/329] 10.222.0.46 () {56 vars in 1099 bytes} [Mon Oct 10 11:46:13 2022] GET /identity/v3/roles/member => generated 84 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:46:13.984673 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a6b376-1c17-4486-9492-eba72d4fc5b6 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:13.992257 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:14.001558 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8b2dfa-d1a1-4e48-808d-d3b7e17b929e None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:14.044184 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b4dd81-9346-44ee-a7ea-a2f1a8cfb27e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:14.044770 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b4dd81-9346-44ee-a7ea-a2f1a8cfb27e None admin] REQUEST_METHOD: `PUT` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:14.045053 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b4dd81-9346-44ee-a7ea-a2f1a8cfb27e None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:14.045441 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b4dd81-9346-44ee-a7ea-a2f1a8cfb27e None admin] PATH_INFO: `/v3/system/users/ea84c3ded2a749c1855718a81537dab9/roles/54c5cfe84bab471db7992eee92787387` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:14.047125 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b4dd81-9346-44ee-a7ea-a2f1a8cfb27e None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=ea84c3ded2a749c1855718a81537dab9, role_id=54c5cfe84bab471db7992eee92787387)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:14.049356 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b4dd81-9346-44ee-a7ea-a2f1a8cfb27e None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:14.092070 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 151/330] 10.222.0.46 () {58 vars in 1334 bytes} [Mon Oct 10 11:46:13 2022] PUT /identity/v3/system/users/ea84c3ded2a749c1855718a81537dab9/roles/54c5cfe84bab471db7992eee92787387 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 11:46:14.093556 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b4dd81-9346-44ee-a7ea-a2f1a8cfb27e None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:14.112703 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a6b376-1c17-4486-9492-eba72d4fc5b6 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:14.137690 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:14.176548 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7353e98a-f73f-4435-972a-e95b58cceacc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:14.177538 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7353e98a-f73f-4435-972a-e95b58cceacc None admin] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:14.178012 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7353e98a-f73f-4435-972a-e95b58cceacc None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:14.178477 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7353e98a-f73f-4435-972a-e95b58cceacc None admin] PATH_INFO: `/v3/groups` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:14.179648 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7353e98a-f73f-4435-972a-e95b58cceacc None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:14.182345 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7353e98a-f73f-4435-972a-e95b58cceacc None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:14.255697 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b4dd81-9346-44ee-a7ea-a2f1a8cfb27e None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:14.298812 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 180/331] 10.222.0.46 () {60 vars in 1134 bytes} [Mon Oct 10 11:46:13 2022] POST /identity/v3/groups => generated 226 bytes in 303 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 11:46:14.309737 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03946c92-ec53-4865-a4f9-c926a0ab863a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:14.310792 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03946c92-ec53-4865-a4f9-c926a0ab863a None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:14.311204 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03946c92-ec53-4865-a4f9-c926a0ab863a None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:14.311660 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03946c92-ec53-4865-a4f9-c926a0ab863a None admin] PATH_INFO: `/v3/roles` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:14.312759 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03946c92-ec53-4865-a4f9-c926a0ab863a None admin] RBAC: Adding query filter params (name=member) {{(pid=81676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:46:14.313278 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03946c92-ec53-4865-a4f9-c926a0ab863a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:14.315746 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03946c92-ec53-4865-a4f9-c926a0ab863a None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:14.332073 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 152/332] 10.222.0.46 () {56 vars in 1101 bytes} [Mon Oct 10 11:46:14 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:46:14.340502 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7353e98a-f73f-4435-972a-e95b58cceacc None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:14.421919 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:14.470486 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7353e98a-f73f-4435-972a-e95b58cceacc None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:14.510182 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b30b66-7389-421b-9927-2cfc10123591 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:14.510899 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b30b66-7389-421b-9927-2cfc10123591 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:14.511176 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b30b66-7389-421b-9927-2cfc10123591 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:14.511444 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b30b66-7389-421b-9927-2cfc10123591 None admin] PATH_INFO: `/v3/users/c5698b4ba6e9482bb7731f8d4b9ef535` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:14.512774 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b30b66-7389-421b-9927-2cfc10123591 None admin] RBAC: Authorizing `identity:get_user(user_id=c5698b4ba6e9482bb7731f8d4b9ef535)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:14.514226 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b30b66-7389-421b-9927-2cfc10123591 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:14.516611 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 181/333] 10.222.0.46 () {56 vars in 1177 bytes} [Mon Oct 10 11:46:14 2022] GET /identity/v3/users/c5698b4ba6e9482bb7731f8d4b9ef535 => generated 293 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:46:14.522397 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03946c92-ec53-4865-a4f9-c926a0ab863a None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:14.690648 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:14.713244 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03946c92-ec53-4865-a4f9-c926a0ab863a None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:14.792025 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74349f51-3105-4cc0-b12d-d2a2209c67ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:14.793109 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74349f51-3105-4cc0-b12d-d2a2209c67ed None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:14.793574 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74349f51-3105-4cc0-b12d-d2a2209c67ed None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:14.794043 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74349f51-3105-4cc0-b12d-d2a2209c67ed None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:14.795585 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74349f51-3105-4cc0-b12d-d2a2209c67ed None admin] RBAC: Adding query filter params (role.id=b2cb2a6e818e47338ce071e6fedfc673, user.id=c5698b4ba6e9482bb7731f8d4b9ef535, scope.system=all) {{(pid=81676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:46:14.796180 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74349f51-3105-4cc0-b12d-d2a2209c67ed None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:14.800052 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74349f51-3105-4cc0-b12d-d2a2209c67ed None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:14.824968 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 153/334] 10.222.0.46 () {56 vars in 1308 bytes} [Mon Oct 10 11:46:14 2022] GET /identity/v3/role_assignments?user.id=c5698b4ba6e9482bb7731f8d4b9ef535&scope.system=all&role.id=b2cb2a6e818e47338ce071e6fedfc673 => generated 510 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 11:46:16.630648 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41f8c5a-8cd7-4be8-82c7-60dee2760db5 None None] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:16.631159 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41f8c5a-8cd7-4be8-82c7-60dee2760db5 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:16.631424 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41f8c5a-8cd7-4be8-82c7-60dee2760db5 None None] PATH_INFO: `/` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:16.632614 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 182/335] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 10 11:46:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:46:16.638446 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fd158c-569b-4c2b-8b68-85f9bd575792 None None] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:16.638938 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fd158c-569b-4c2b-8b68-85f9bd575792 None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:16.639038 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fd158c-569b-4c2b-8b68-85f9bd575792 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:16.663030 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.auth.core [None req-a1fd158c-569b-4c2b-8b68-85f9bd575792 None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:46:16.664232 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-a1fd158c-569b-4c2b-8b68-85f9bd575792 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:46:16.748790 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 154/336] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 10 11:46:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 10 11:46:17.310210 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41f8c5a-8cd7-4be8-82c7-60dee2760db5 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:17.439278 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41f8c5a-8cd7-4be8-82c7-60dee2760db5 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:17.475829 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1ef7d1-d47a-49f1-aa23-6c654c2a6e90 None None] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:17.476418 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1ef7d1-d47a-49f1-aa23-6c654c2a6e90 None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:17.476869 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1ef7d1-d47a-49f1-aa23-6c654c2a6e90 None None] PATH_INFO: `/` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:17.478412 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 155/337] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 10 11:46:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 10 11:46:17.486949 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b00dee-6897-44ce-9c18-cd378bb73ffa None None] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:17.489522 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b00dee-6897-44ce-9c18-cd378bb73ffa None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:17.489522 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b00dee-6897-44ce-9c18-cd378bb73ffa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:17.503946 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:17.513833 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26b23cc-9194-4969-bd6a-8a554bfb8e4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:17.514495 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26b23cc-9194-4969-bd6a-8a554bfb8e4e None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:17.514813 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26b23cc-9194-4969-bd6a-8a554bfb8e4e None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:17.515093 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26b23cc-9194-4969-bd6a-8a554bfb8e4e None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:17.539760 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26b23cc-9194-4969-bd6a-8a554bfb8e4e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 10 11:46:17.544975 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application [None req-e26b23cc-9194-4969-bd6a-8a554bfb8e4e None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 10 11:46:17.544975 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:46:17.544975 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:46:17.544975 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:46:17.544975 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:46:17.544975 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:46:17.544975 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:46:17.544975 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:46:17.544975 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:46:17.544975 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:46:17.544975 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:46:17.544975 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:46:17.544975 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 10 11:46:17.544975 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 10 11:46:17.544975 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 10 11:46:17.544975 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 10 11:46:17.545913 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 10 11:46:17.545913 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 10 11:46:17.545913 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 10 11:46:17.545913 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 10 11:46:17.545913 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:46:17.545913 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:46:17.545913 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:46:17.545913 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:46:17.545913 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:46:17.545913 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:46:17.545913 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:46:17.545913 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application with Lock( Oct 10 11:46:17.545913 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:46:17.545913 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:46:17.545913 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:46:17.545913 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:46:17.546818 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:46:17.546818 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:46:17.546818 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:46:17.546818 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:46:17.546818 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 10 11:46:17.546818 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 10 11:46:17.546818 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 10 11:46:17.546818 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 10 11:46:17.546818 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 10 11:46:17.546818 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 10 11:46:17.546818 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 10 11:46:17.546818 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Oct 10 11:46:17.552415 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 183/338] 10.222.0.46 () {56 vars in 1099 bytes} [Mon Oct 10 11:46:17 2022] GET /identity/v3/roles/reader => generated 84 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:46:17.557813 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26b23cc-9194-4969-bd6a-8a554bfb8e4e None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:17.600792 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.auth.core [None req-d8b00dee-6897-44ce-9c18-cd378bb73ffa None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:46:17.602212 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-d8b00dee-6897-44ce-9c18-cd378bb73ffa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:46:17.698530 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 156/339] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 10 11:46:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 10 11:46:17.700967 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed42ba0d-29e5-4629-9801-69f163a49e3d None None] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:17.701508 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed42ba0d-29e5-4629-9801-69f163a49e3d None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:17.702009 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed42ba0d-29e5-4629-9801-69f163a49e3d None None] PATH_INFO: `/` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:17.704841 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 157/340] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 10 11:46:17 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:46:17.711606 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb01d3be-ba49-4144-ad71-97f42ac22921 None None] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:17.712223 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb01d3be-ba49-4144-ad71-97f42ac22921 None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:17.712675 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb01d3be-ba49-4144-ad71-97f42ac22921 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:17.736158 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.auth.core [None req-fb01d3be-ba49-4144-ad71-97f42ac22921 None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:46:17.737726 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-fb01d3be-ba49-4144-ad71-97f42ac22921 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:46:17.739989 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26b23cc-9194-4969-bd6a-8a554bfb8e4e None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:17.740820 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:17.814244 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228b407c-f2a7-4ce5-bcdf-a9a13f50d973 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:17.815190 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228b407c-f2a7-4ce5-bcdf-a9a13f50d973 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:17.815571 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228b407c-f2a7-4ce5-bcdf-a9a13f50d973 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:17.815945 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228b407c-f2a7-4ce5-bcdf-a9a13f50d973 None admin] PATH_INFO: `/v3/roles` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:17.816819 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228b407c-f2a7-4ce5-bcdf-a9a13f50d973 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:46:17.817262 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228b407c-f2a7-4ce5-bcdf-a9a13f50d973 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:17.822944 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228b407c-f2a7-4ce5-bcdf-a9a13f50d973 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:17.830712 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 158/341] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 10 11:46:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 10 11:46:17.838584 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 184/342] 10.222.0.46 () {56 vars in 1101 bytes} [Mon Oct 10 11:46:17 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 11:46:17.845489 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb01d3be-ba49-4144-ad71-97f42ac22921 None None] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:17.981418 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb01d3be-ba49-4144-ad71-97f42ac22921 None None] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:18.027161 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00e725a-1e3f-426a-8511-4f785dac8c4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:18.027794 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00e725a-1e3f-426a-8511-4f785dac8c4a None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:18.028075 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00e725a-1e3f-426a-8511-4f785dac8c4a None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:18.028431 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00e725a-1e3f-426a-8511-4f785dac8c4a None admin] PATH_INFO: `/v3/users/ea84c3ded2a749c1855718a81537dab9` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:18.029816 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00e725a-1e3f-426a-8511-4f785dac8c4a None admin] RBAC: Authorizing `identity:get_user(user_id=ea84c3ded2a749c1855718a81537dab9)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:18.031465 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00e725a-1e3f-426a-8511-4f785dac8c4a None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:18.034185 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 159/343] 10.222.0.46 () {56 vars in 1177 bytes} [Mon Oct 10 11:46:17 2022] GET /identity/v3/users/ea84c3ded2a749c1855718a81537dab9 => generated 293 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:46:18.040437 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228b407c-f2a7-4ce5-bcdf-a9a13f50d973 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:18.108605 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:18.117552 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00e725a-1e3f-426a-8511-4f785dac8c4a None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:18.127347 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:18.169164 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228b407c-f2a7-4ce5-bcdf-a9a13f50d973 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:18.213269 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724ced5f-63c5-4b4b-bea1-c18866d3215a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:18.213953 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724ced5f-63c5-4b4b-bea1-c18866d3215a None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:18.214227 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724ced5f-63c5-4b4b-bea1-c18866d3215a None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:18.214660 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724ced5f-63c5-4b4b-bea1-c18866d3215a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:18.215975 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724ced5f-63c5-4b4b-bea1-c18866d3215a None admin] RBAC: Adding query filter params (role.id=54c5cfe84bab471db7992eee92787387, user.id=ea84c3ded2a749c1855718a81537dab9, scope.system=all) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:46:18.216435 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724ced5f-63c5-4b4b-bea1-c18866d3215a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:18.218336 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724ced5f-63c5-4b4b-bea1-c18866d3215a None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:18.233798 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 185/344] 10.222.0.46 () {56 vars in 1308 bytes} [Mon Oct 10 11:46:18 2022] GET /identity/v3/role_assignments?user.id=ea84c3ded2a749c1855718a81537dab9&scope.system=all&role.id=54c5cfe84bab471db7992eee92787387 => generated 510 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:46:18.300811 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dba4455-3ca1-415d-8116-2d5fbce6667c None None] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:18.301552 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dba4455-3ca1-415d-8116-2d5fbce6667c None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:18.302030 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dba4455-3ca1-415d-8116-2d5fbce6667c None None] PATH_INFO: `/` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:18.304369 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 186/345] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 10 11:46:18 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 10 11:46:18.310194 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683aefa9-26bd-42ed-952d-255b88c681ad None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:18.310805 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683aefa9-26bd-42ed-952d-255b88c681ad None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:18.311268 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683aefa9-26bd-42ed-952d-255b88c681ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:18.327114 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00e725a-1e3f-426a-8511-4f785dac8c4a None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:18.336911 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-683aefa9-26bd-42ed-952d-255b88c681ad None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:46:18.338418 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-683aefa9-26bd-42ed-952d-255b88c681ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:46:18.419614 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ebfdbc-f423-43be-ae30-25bd51b7731b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:18.420286 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ebfdbc-f423-43be-ae30-25bd51b7731b None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:18.420568 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ebfdbc-f423-43be-ae30-25bd51b7731b None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:18.420841 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ebfdbc-f423-43be-ae30-25bd51b7731b None admin] PATH_INFO: `/v3/roles/member` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:18.432706 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 187/346] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 10 11:46:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 10 11:46:18.440095 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ebfdbc-f423-43be-ae30-25bd51b7731b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 10 11:46:18.446118 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application [None req-74ebfdbc-f423-43be-ae30-25bd51b7731b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 10 11:46:18.446118 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:46:18.446118 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:46:18.446118 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:46:18.446118 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:46:18.446118 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:46:18.446118 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:46:18.446118 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:46:18.446118 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:46:18.446118 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:46:18.446118 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:46:18.446118 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:46:18.446118 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 10 11:46:18.446118 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 10 11:46:18.446118 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 10 11:46:18.446118 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 10 11:46:18.447098 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 10 11:46:18.447098 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 10 11:46:18.447098 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 10 11:46:18.447098 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 10 11:46:18.447098 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:46:18.447098 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:46:18.447098 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:46:18.447098 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:46:18.447098 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:46:18.447098 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:46:18.447098 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:46:18.447098 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application with Lock( Oct 10 11:46:18.447098 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:46:18.447098 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:46:18.447098 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:46:18.447098 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:46:18.447890 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:46:18.447890 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:46:18.447890 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:46:18.447890 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:46:18.447890 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 10 11:46:18.447890 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 10 11:46:18.447890 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 10 11:46:18.447890 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 10 11:46:18.447890 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 10 11:46:18.447890 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 10 11:46:18.447890 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 10 11:46:18.447890 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Oct 10 11:46:18.450503 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 160/347] 10.222.0.46 () {56 vars in 1099 bytes} [Mon Oct 10 11:46:18 2022] GET /identity/v3/roles/member => generated 84 bytes in 333 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 10 11:46:18.460686 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683aefa9-26bd-42ed-952d-255b88c681ad None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:18.522469 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:18.528367 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ebfdbc-f423-43be-ae30-25bd51b7731b None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:18.622718 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683aefa9-26bd-42ed-952d-255b88c681ad None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:18.656257 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:18.686435 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3340d69-bfa8-49a0-8997-6f8ed51143da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:18.687104 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3340d69-bfa8-49a0-8997-6f8ed51143da None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:18.687416 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3340d69-bfa8-49a0-8997-6f8ed51143da None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:18.688227 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3340d69-bfa8-49a0-8997-6f8ed51143da None admin] PATH_INFO: `/v3/roles` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:18.690530 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3340d69-bfa8-49a0-8997-6f8ed51143da None admin] RBAC: Adding query filter params (name=member) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:46:18.690530 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3340d69-bfa8-49a0-8997-6f8ed51143da None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:18.692306 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3340d69-bfa8-49a0-8997-6f8ed51143da None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:18.711873 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 188/348] 10.222.0.46 () {56 vars in 1101 bytes} [Mon Oct 10 11:46:18 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 11:46:18.720776 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3340d69-bfa8-49a0-8997-6f8ed51143da None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:18.721654 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ebfdbc-f423-43be-ae30-25bd51b7731b None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:18.777236 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3c6934-7166-49db-8b62-645407559b70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:18.777788 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3c6934-7166-49db-8b62-645407559b70 None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:18.778072 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3c6934-7166-49db-8b62-645407559b70 None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:18.778351 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3c6934-7166-49db-8b62-645407559b70 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:18.794738 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3c6934-7166-49db-8b62-645407559b70 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 10 11:46:18.803893 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application [None req-2d3c6934-7166-49db-8b62-645407559b70 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 10 11:46:18.803893 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:46:18.803893 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:46:18.803893 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:46:18.803893 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:46:18.803893 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:46:18.803893 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:46:18.803893 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:46:18.803893 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:46:18.803893 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:46:18.803893 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:46:18.803893 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:46:18.803893 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 10 11:46:18.803893 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 10 11:46:18.803893 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 10 11:46:18.803893 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 10 11:46:18.804994 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 10 11:46:18.804994 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 10 11:46:18.804994 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 10 11:46:18.804994 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 10 11:46:18.804994 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:46:18.804994 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:46:18.804994 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:46:18.804994 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:46:18.804994 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:46:18.804994 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:46:18.804994 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:46:18.804994 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application with Lock( Oct 10 11:46:18.804994 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:46:18.804994 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:46:18.804994 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:46:18.804994 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:46:18.805874 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:46:18.805874 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:46:18.805874 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:46:18.805874 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:46:18.805874 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 10 11:46:18.805874 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 10 11:46:18.805874 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 10 11:46:18.805874 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 10 11:46:18.805874 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 10 11:46:18.805874 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 10 11:46:18.805874 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 10 11:46:18.805874 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Oct 10 11:46:18.805874 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 161/349] 10.222.0.46 () {56 vars in 1099 bytes} [Mon Oct 10 11:46:18 2022] GET /identity/v3/roles/member => generated 84 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 10 11:46:18.811123 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3c6934-7166-49db-8b62-645407559b70 None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:18.901132 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3340d69-bfa8-49a0-8997-6f8ed51143da None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:18.905664 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:18.957383 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e7e471-75ca-42ee-a2d6-496aab9bcaea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:18.958018 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e7e471-75ca-42ee-a2d6-496aab9bcaea None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:18.958301 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e7e471-75ca-42ee-a2d6-496aab9bcaea None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:18.958576 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e7e471-75ca-42ee-a2d6-496aab9bcaea None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:18.968454 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3c6934-7166-49db-8b62-645407559b70 None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:19.002589 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:19.014238 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e27901-d553-44d7-90b7-a6108cb7d5b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:19.017691 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e27901-d553-44d7-90b7-a6108cb7d5b7 None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:19.017691 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e27901-d553-44d7-90b7-a6108cb7d5b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:19.017691 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e27901-d553-44d7-90b7-a6108cb7d5b7 None admin] PATH_INFO: `/v3/roles` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:19.017691 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e27901-d553-44d7-90b7-a6108cb7d5b7 None admin] RBAC: Adding query filter params (name=member) {{(pid=81676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:46:19.018647 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e27901-d553-44d7-90b7-a6108cb7d5b7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:19.020947 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e27901-d553-44d7-90b7-a6108cb7d5b7 None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:19.035527 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 162/350] 10.222.0.46 () {56 vars in 1101 bytes} [Mon Oct 10 11:46:18 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:46:19.038187 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e7e471-75ca-42ee-a2d6-496aab9bcaea None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:19.039735 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e7e471-75ca-42ee-a2d6-496aab9bcaea None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:19.041160 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e27901-d553-44d7-90b7-a6108cb7d5b7 None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:19.168553 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 189/351] 10.222.0.46 () {58 vars in 1282 bytes} [Mon Oct 10 11:46:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 448 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Oct 10 11:46:19.169420 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfd61f7-f90e-4505-a1d2-f52a864c49ca None None] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:19.169832 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfd61f7-f90e-4505-a1d2-f52a864c49ca None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:19.170122 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfd61f7-f90e-4505-a1d2-f52a864c49ca None None] PATH_INFO: `/` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:19.171428 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 190/352] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 10 11:46:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:46:19.172759 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfd61f7-f90e-4505-a1d2-f52a864c49ca None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:19.183822 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:19.220720 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e27901-d553-44d7-90b7-a6108cb7d5b7 None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:19.272587 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ab49af-00d4-43f8-af4e-8d2b86d385ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:19.272587 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ab49af-00d4-43f8-af4e-8d2b86d385ef None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:19.272587 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ab49af-00d4-43f8-af4e-8d2b86d385ef None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:19.272587 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ab49af-00d4-43f8-af4e-8d2b86d385ef None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:19.345180 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfd61f7-f90e-4505-a1d2-f52a864c49ca None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:19.361039 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:19.379054 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ab49af-00d4-43f8-af4e-8d2b86d385ef None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:19.383848 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ab49af-00d4-43f8-af4e-8d2b86d385ef None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:19.405578 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36251650-181a-4f4f-b0fa-6395bc659ae9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:19.406233 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36251650-181a-4f4f-b0fa-6395bc659ae9 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:19.407073 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36251650-181a-4f4f-b0fa-6395bc659ae9 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:19.407347 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36251650-181a-4f4f-b0fa-6395bc659ae9 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:19.425623 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36251650-181a-4f4f-b0fa-6395bc659ae9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 10 11:46:19.430693 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application [None req-36251650-181a-4f4f-b0fa-6395bc659ae9 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 10 11:46:19.430693 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:46:19.430693 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:46:19.430693 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:46:19.430693 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:46:19.430693 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:46:19.430693 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:46:19.430693 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:46:19.430693 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:46:19.430693 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:46:19.430693 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:46:19.430693 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:46:19.430693 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 10 11:46:19.430693 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 10 11:46:19.430693 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 10 11:46:19.430693 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 10 11:46:19.431857 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 10 11:46:19.431857 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 10 11:46:19.431857 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 10 11:46:19.431857 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 10 11:46:19.431857 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:46:19.431857 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:46:19.431857 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:46:19.431857 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:46:19.431857 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:46:19.431857 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:46:19.431857 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:46:19.431857 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application with Lock( Oct 10 11:46:19.431857 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:46:19.431857 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:46:19.431857 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:46:19.431857 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:46:19.432798 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:46:19.432798 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:46:19.432798 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:46:19.432798 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:46:19.432798 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 10 11:46:19.432798 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 10 11:46:19.432798 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 10 11:46:19.432798 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 10 11:46:19.432798 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 10 11:46:19.432798 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 10 11:46:19.432798 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 10 11:46:19.432798 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Oct 10 11:46:19.432798 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 191/353] 10.222.0.46 () {56 vars in 1114 bytes} [Mon Oct 10 11:46:19 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:46:19.433844 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5bf3c3-a073-4e6a-92c0-08caa2541c73 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:19.433844 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5bf3c3-a073-4e6a-92c0-08caa2541c73 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:19.433844 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5bf3c3-a073-4e6a-92c0-08caa2541c73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:19.458748 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-6d5bf3c3-a073-4e6a-92c0-08caa2541c73 None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:46:19.459959 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-6d5bf3c3-a073-4e6a-92c0-08caa2541c73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:46:19.532642 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 163/354] 10.222.0.46 () {58 vars in 1282 bytes} [Mon Oct 10 11:46:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 492 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 10 11:46:19.533577 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ab49af-00d4-43f8-af4e-8d2b86d385ef None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:19.546082 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:19.562263 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 192/355] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 10 11:46:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 10 11:46:19.565878 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5bf3c3-a073-4e6a-92c0-08caa2541c73 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:19.659199 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:19.677081 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ab49af-00d4-43f8-af4e-8d2b86d385ef None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:19.729484 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749fe761-fa98-4f01-be83-4dabf9447326 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:19.731467 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749fe761-fa98-4f01-be83-4dabf9447326 None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:19.732327 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749fe761-fa98-4f01-be83-4dabf9447326 None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:19.732837 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749fe761-fa98-4f01-be83-4dabf9447326 None admin] PATH_INFO: `/v3/projects/2d0af77a941540ef9b78b89ebae1ef1a` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:19.735346 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749fe761-fa98-4f01-be83-4dabf9447326 None admin] RBAC: Authorizing `identity:get_project(project_id=2d0af77a941540ef9b78b89ebae1ef1a)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:19.738243 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749fe761-fa98-4f01-be83-4dabf9447326 None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:19.740741 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 164/356] 10.222.0.46 () {56 vars in 1186 bytes} [Mon Oct 10 11:46:19 2022] GET /identity/v3/projects/2d0af77a941540ef9b78b89ebae1ef1a => generated 298 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:46:19.741440 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5bf3c3-a073-4e6a-92c0-08caa2541c73 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:19.742247 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749fe761-fa98-4f01-be83-4dabf9447326 None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:19.790471 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0102f98-6cce-4ebd-8f13-cc50ea096613 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:19.791216 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0102f98-6cce-4ebd-8f13-cc50ea096613 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:19.791509 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0102f98-6cce-4ebd-8f13-cc50ea096613 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:19.791781 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0102f98-6cce-4ebd-8f13-cc50ea096613 None admin] PATH_INFO: `/v3/roles` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:19.792500 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0102f98-6cce-4ebd-8f13-cc50ea096613 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:46:19.792812 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0102f98-6cce-4ebd-8f13-cc50ea096613 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:19.794390 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0102f98-6cce-4ebd-8f13-cc50ea096613 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:19.806937 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 193/357] 10.222.0.46 () {56 vars in 1111 bytes} [Mon Oct 10 11:46:19 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:46:19.807878 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0102f98-6cce-4ebd-8f13-cc50ea096613 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:19.890058 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:19.919871 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749fe761-fa98-4f01-be83-4dabf9447326 None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:19.957619 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0102f98-6cce-4ebd-8f13-cc50ea096613 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:19.981935 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3b07bc-5d90-4031-8d8f-8076b073893c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:19.982490 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3b07bc-5d90-4031-8d8f-8076b073893c None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:19.985062 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3b07bc-5d90-4031-8d8f-8076b073893c None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:19.985569 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3b07bc-5d90-4031-8d8f-8076b073893c None admin] PATH_INFO: `/v3/projects/55927449396d4c4f8470544dd7a23ed9` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:19.987304 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3b07bc-5d90-4031-8d8f-8076b073893c None admin] RBAC: Authorizing `identity:get_project(project_id=55927449396d4c4f8470544dd7a23ed9)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:19.989062 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3b07bc-5d90-4031-8d8f-8076b073893c None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:19.991985 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 165/358] 10.222.0.46 () {56 vars in 1186 bytes} [Mon Oct 10 11:46:19 2022] GET /identity/v3/projects/55927449396d4c4f8470544dd7a23ed9 => generated 302 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:46:19.992655 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3b07bc-5d90-4031-8d8f-8076b073893c None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:20.014159 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45454225-cb2d-4e92-95d4-05835e5275c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:20.014910 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45454225-cb2d-4e92-95d4-05835e5275c1 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:20.015200 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45454225-cb2d-4e92-95d4-05835e5275c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:20.015515 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45454225-cb2d-4e92-95d4-05835e5275c1 None admin] PATH_INFO: `/v3/groups/3ec08ca1a9da4374b4f87de87f03b1f4` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:20.030545 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45454225-cb2d-4e92-95d4-05835e5275c1 None admin] RBAC: Authorizing `identity:get_group(group_id=3ec08ca1a9da4374b4f87de87f03b1f4)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:20.032225 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:20.034111 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45454225-cb2d-4e92-95d4-05835e5275c1 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:20.037133 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 194/359] 10.222.0.46 () {56 vars in 1180 bytes} [Mon Oct 10 11:46:19 2022] GET /identity/v3/groups/3ec08ca1a9da4374b4f87de87f03b1f4 => generated 226 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:46:20.037901 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45454225-cb2d-4e92-95d4-05835e5275c1 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:20.135629 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3b07bc-5d90-4031-8d8f-8076b073893c None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:20.177008 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45454225-cb2d-4e92-95d4-05835e5275c1 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:20.182140 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1deeb3c-9bd0-48c1-985f-c29ca9926a0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:20.182795 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1deeb3c-9bd0-48c1-985f-c29ca9926a0f None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:20.183084 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1deeb3c-9bd0-48c1-985f-c29ca9926a0f None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:20.183363 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1deeb3c-9bd0-48c1-985f-c29ca9926a0f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:20.199740 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:20.219479 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1092dd-2ed8-451a-aae8-353647e29f37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:20.220134 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1092dd-2ed8-451a-aae8-353647e29f37 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:20.220412 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1092dd-2ed8-451a-aae8-353647e29f37 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:20.220995 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1092dd-2ed8-451a-aae8-353647e29f37 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:20.232696 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1092dd-2ed8-451a-aae8-353647e29f37 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 10 11:46:20.239586 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application [None req-1f1092dd-2ed8-451a-aae8-353647e29f37 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 10 11:46:20.239586 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:46:20.239586 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:46:20.239586 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:46:20.239586 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:46:20.239586 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:46:20.239586 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:46:20.239586 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:46:20.239586 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:46:20.239586 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:46:20.239586 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:46:20.239586 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:46:20.239586 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 10 11:46:20.239586 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 10 11:46:20.239586 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 10 11:46:20.239586 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 10 11:46:20.240721 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 10 11:46:20.240721 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 10 11:46:20.240721 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 10 11:46:20.240721 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 10 11:46:20.240721 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:46:20.240721 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:46:20.240721 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:46:20.240721 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:46:20.240721 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:46:20.240721 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:46:20.240721 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:46:20.240721 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application with Lock( Oct 10 11:46:20.240721 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:46:20.240721 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:46:20.240721 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:46:20.240721 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:46:20.241713 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:46:20.241713 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:46:20.241713 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:46:20.241713 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:46:20.241713 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 10 11:46:20.241713 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 10 11:46:20.241713 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 10 11:46:20.241713 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 10 11:46:20.241713 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 10 11:46:20.241713 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 10 11:46:20.241713 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 10 11:46:20.241713 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Oct 10 11:46:20.242833 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 195/360] 10.222.0.46 () {56 vars in 1114 bytes} [Mon Oct 10 11:46:20 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:46:20.244059 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1092dd-2ed8-451a-aae8-353647e29f37 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:20.270351 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1deeb3c-9bd0-48c1-985f-c29ca9926a0f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:20.271891 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1deeb3c-9bd0-48c1-985f-c29ca9926a0f None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:20.300568 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:20.368890 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1092dd-2ed8-451a-aae8-353647e29f37 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:20.392509 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 166/361] 10.222.0.46 () {58 vars in 1282 bytes} [Mon Oct 10 11:46:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 10 11:46:20.393621 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1deeb3c-9bd0-48c1-985f-c29ca9926a0f None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:20.412282 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a908e20-8105-42e9-a42f-41f79292e298 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:20.415844 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a908e20-8105-42e9-a42f-41f79292e298 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:20.415844 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a908e20-8105-42e9-a42f-41f79292e298 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:20.415844 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a908e20-8105-42e9-a42f-41f79292e298 None admin] PATH_INFO: `/v3/groups/3ec08ca1a9da4374b4f87de87f03b1f4` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:20.415844 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a908e20-8105-42e9-a42f-41f79292e298 None admin] RBAC: Authorizing `identity:get_group(group_id=3ec08ca1a9da4374b4f87de87f03b1f4)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:20.416859 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a908e20-8105-42e9-a42f-41f79292e298 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:20.418711 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 196/362] 10.222.0.46 () {56 vars in 1180 bytes} [Mon Oct 10 11:46:20 2022] GET /identity/v3/groups/3ec08ca1a9da4374b4f87de87f03b1f4 => generated 226 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:46:20.422690 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a65ffe-5cc3-4588-afe9-d7c87722e99b None None] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:20.422690 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a65ffe-5cc3-4588-afe9-d7c87722e99b None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:20.422690 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a65ffe-5cc3-4588-afe9-d7c87722e99b None None] PATH_INFO: `/` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:20.422690 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 197/363] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 10 11:46:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:46:20.423030 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a65ffe-5cc3-4588-afe9-d7c87722e99b None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:20.487349 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:20.534783 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1deeb3c-9bd0-48c1-985f-c29ca9926a0f None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:20.544721 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a65ffe-5cc3-4588-afe9-d7c87722e99b None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:20.561267 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:20.575412 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64ec725-4e60-4aef-ad03-3b1d8f198e68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:20.576032 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64ec725-4e60-4aef-ad03-3b1d8f198e68 None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:20.576314 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64ec725-4e60-4aef-ad03-3b1d8f198e68 None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:20.576590 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64ec725-4e60-4aef-ad03-3b1d8f198e68 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:20.577336 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64ec725-4e60-4aef-ad03-3b1d8f198e68 None admin] RBAC: Adding query filter params (group.id=3ec08ca1a9da4374b4f87de87f03b1f4, role.id=b2cb2a6e818e47338ce071e6fedfc673, scope.project.id=2d0af77a941540ef9b78b89ebae1ef1a) {{(pid=81676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:46:20.577645 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64ec725-4e60-4aef-ad03-3b1d8f198e68 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:20.579661 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64ec725-4e60-4aef-ad03-3b1d8f198e68 None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:20.585813 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 167/364] 10.222.0.46 () {56 vars in 1376 bytes} [Mon Oct 10 11:46:20 2022] GET /identity/v3/role_assignments?group.id=3ec08ca1a9da4374b4f87de87f03b1f4&scope.project.id=2d0af77a941540ef9b78b89ebae1ef1a&role.id=b2cb2a6e818e47338ce071e6fedfc673 => generated 260 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:46:20.586574 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64ec725-4e60-4aef-ad03-3b1d8f198e68 None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:20.588896 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ae5f79-4037-4fc0-80eb-2a364d1035af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:20.589728 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ae5f79-4037-4fc0-80eb-2a364d1035af None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:20.590175 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ae5f79-4037-4fc0-80eb-2a364d1035af None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:20.590454 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ae5f79-4037-4fc0-80eb-2a364d1035af None admin] PATH_INFO: `/v3/roles` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:20.591182 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ae5f79-4037-4fc0-80eb-2a364d1035af None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:46:20.591490 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ae5f79-4037-4fc0-80eb-2a364d1035af None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:20.593145 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ae5f79-4037-4fc0-80eb-2a364d1035af None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:20.604504 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 198/365] 10.222.0.46 () {56 vars in 1111 bytes} [Mon Oct 10 11:46:20 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:46:20.610483 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ae5f79-4037-4fc0-80eb-2a364d1035af None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:20.746770 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64ec725-4e60-4aef-ad03-3b1d8f198e68 None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:20.750048 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ae5f79-4037-4fc0-80eb-2a364d1035af None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:20.764926 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:20.797110 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7d06d3-d5fb-4ff4-ac1f-e38d7f4790d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:20.797910 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7d06d3-d5fb-4ff4-ac1f-e38d7f4790d6 None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:20.798212 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7d06d3-d5fb-4ff4-ac1f-e38d7f4790d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:20.798493 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7d06d3-d5fb-4ff4-ac1f-e38d7f4790d6 None admin] PATH_INFO: `/v3/projects/2d0af77a941540ef9b78b89ebae1ef1a` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:20.799752 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7d06d3-d5fb-4ff4-ac1f-e38d7f4790d6 None admin] RBAC: Authorizing `identity:get_project(project_id=2d0af77a941540ef9b78b89ebae1ef1a)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:20.801204 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7d06d3-d5fb-4ff4-ac1f-e38d7f4790d6 None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:20.803087 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 168/366] 10.222.0.46 () {56 vars in 1186 bytes} [Mon Oct 10 11:46:20 2022] GET /identity/v3/projects/2d0af77a941540ef9b78b89ebae1ef1a => generated 298 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:46:20.804224 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1aad81-4db2-4b57-b7ce-286220640efb None None] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:20.804571 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1aad81-4db2-4b57-b7ce-286220640efb None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:20.804751 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1aad81-4db2-4b57-b7ce-286220640efb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:20.805331 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e882bbdb-cb05-4fc7-b077-f5e5f2a432c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:20.806071 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e882bbdb-cb05-4fc7-b077-f5e5f2a432c8 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:20.806550 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e882bbdb-cb05-4fc7-b077-f5e5f2a432c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:20.806885 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e882bbdb-cb05-4fc7-b077-f5e5f2a432c8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:20.807612 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e882bbdb-cb05-4fc7-b077-f5e5f2a432c8 None admin] RBAC: Adding query filter params (group.id=3ec08ca1a9da4374b4f87de87f03b1f4, role.id=b2cb2a6e818e47338ce071e6fedfc673, scope.project.id=55927449396d4c4f8470544dd7a23ed9) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:46:20.807920 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e882bbdb-cb05-4fc7-b077-f5e5f2a432c8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:20.809326 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e882bbdb-cb05-4fc7-b077-f5e5f2a432c8 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:20.816532 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 199/367] 10.222.0.46 () {56 vars in 1376 bytes} [Mon Oct 10 11:46:20 2022] GET /identity/v3/role_assignments?group.id=3ec08ca1a9da4374b4f87de87f03b1f4&scope.project.id=55927449396d4c4f8470544dd7a23ed9&role.id=b2cb2a6e818e47338ce071e6fedfc673 => generated 260 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:46:20.817163 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e882bbdb-cb05-4fc7-b077-f5e5f2a432c8 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:20.826549 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.auth.core [None req-1c1aad81-4db2-4b57-b7ce-286220640efb None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:46:20.828027 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-1c1aad81-4db2-4b57-b7ce-286220640efb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:46:20.902751 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:20.926829 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 169/368] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 10 11:46:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 10 11:46:20.928778 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1aad81-4db2-4b57-b7ce-286220640efb None None] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:20.971761 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e882bbdb-cb05-4fc7-b077-f5e5f2a432c8 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:21.044095 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05681aa-b989-4ec6-aa3a-325f86e43cc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:21.044644 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05681aa-b989-4ec6-aa3a-325f86e43cc9 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:21.044924 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05681aa-b989-4ec6-aa3a-325f86e43cc9 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:21.045199 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05681aa-b989-4ec6-aa3a-325f86e43cc9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:21.130424 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:21.135091 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1aad81-4db2-4b57-b7ce-286220640efb None None] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:21.187235 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05681aa-b989-4ec6-aa3a-325f86e43cc9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:21.188941 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05681aa-b989-4ec6-aa3a-325f86e43cc9 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:21.203986 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17437f3f-41f8-4022-bf6f-e1b48d4501fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:21.204652 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17437f3f-41f8-4022-bf6f-e1b48d4501fb None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:21.205009 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17437f3f-41f8-4022-bf6f-e1b48d4501fb None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:21.205269 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17437f3f-41f8-4022-bf6f-e1b48d4501fb None admin] PATH_INFO: `/v3/groups/3ec08ca1a9da4374b4f87de87f03b1f4` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:21.206485 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17437f3f-41f8-4022-bf6f-e1b48d4501fb None admin] RBAC: Authorizing `identity:get_group(group_id=3ec08ca1a9da4374b4f87de87f03b1f4)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:21.207959 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17437f3f-41f8-4022-bf6f-e1b48d4501fb None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:21.209923 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 170/369] 10.222.0.46 () {56 vars in 1180 bytes} [Mon Oct 10 11:46:20 2022] GET /identity/v3/groups/3ec08ca1a9da4374b4f87de87f03b1f4 => generated 226 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:46:21.215614 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17437f3f-41f8-4022-bf6f-e1b48d4501fb None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:21.318132 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 200/370] 10.222.0.46 () {58 vars in 1282 bytes} [Mon Oct 10 11:46:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 501 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 10 11:46:21.318960 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05681aa-b989-4ec6-aa3a-325f86e43cc9 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:21.332101 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:21.402083 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17437f3f-41f8-4022-bf6f-e1b48d4501fb None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:21.427486 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:21.450709 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893bc64e-36a7-44cd-9f75-1ddd0715bc8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:21.451354 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893bc64e-36a7-44cd-9f75-1ddd0715bc8a None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:21.452034 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893bc64e-36a7-44cd-9f75-1ddd0715bc8a None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:21.452314 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893bc64e-36a7-44cd-9f75-1ddd0715bc8a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:21.453884 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893bc64e-36a7-44cd-9f75-1ddd0715bc8a None admin] RBAC: Adding query filter params (group.id=3ec08ca1a9da4374b4f87de87f03b1f4, role.id=6762beebc2bf4e6f804cbbe55a475e70, scope.project.id=2d0af77a941540ef9b78b89ebae1ef1a) {{(pid=81676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:46:21.454200 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893bc64e-36a7-44cd-9f75-1ddd0715bc8a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:21.456370 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893bc64e-36a7-44cd-9f75-1ddd0715bc8a None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:21.464064 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 171/371] 10.222.0.46 () {56 vars in 1376 bytes} [Mon Oct 10 11:46:21 2022] GET /identity/v3/role_assignments?group.id=3ec08ca1a9da4374b4f87de87f03b1f4&scope.project.id=2d0af77a941540ef9b78b89ebae1ef1a&role.id=6762beebc2bf4e6f804cbbe55a475e70 => generated 260 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 11:46:21.465035 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893bc64e-36a7-44cd-9f75-1ddd0715bc8a None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:21.466343 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05681aa-b989-4ec6-aa3a-325f86e43cc9 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:21.521241 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d13cbb-eece-4535-b256-40ef9174f605 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:21.522010 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d13cbb-eece-4535-b256-40ef9174f605 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:21.522353 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d13cbb-eece-4535-b256-40ef9174f605 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:21.522758 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d13cbb-eece-4535-b256-40ef9174f605 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:21.540672 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d13cbb-eece-4535-b256-40ef9174f605 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 10 11:46:21.546299 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application [None req-31d13cbb-eece-4535-b256-40ef9174f605 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 10 11:46:21.546299 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:46:21.546299 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:46:21.546299 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:46:21.546299 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:46:21.546299 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:46:21.546299 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:46:21.546299 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:46:21.546299 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:46:21.546299 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:46:21.546299 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:46:21.546299 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:46:21.546299 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 10 11:46:21.546299 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 10 11:46:21.546299 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 10 11:46:21.546299 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 10 11:46:21.547943 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 10 11:46:21.547943 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 10 11:46:21.547943 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 10 11:46:21.547943 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 10 11:46:21.547943 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:46:21.547943 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:46:21.547943 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:46:21.547943 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:46:21.547943 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:46:21.547943 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:46:21.547943 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:46:21.547943 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application with Lock( Oct 10 11:46:21.547943 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:46:21.547943 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:46:21.547943 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:46:21.547943 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:46:21.548703 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:46:21.548703 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:46:21.548703 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:46:21.548703 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:46:21.548703 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 10 11:46:21.548703 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 10 11:46:21.548703 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 10 11:46:21.548703 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 10 11:46:21.548703 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 10 11:46:21.548703 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 10 11:46:21.548703 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 10 11:46:21.548703 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Oct 10 11:46:21.553540 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 201/372] 10.222.0.46 () {56 vars in 1096 bytes} [Mon Oct 10 11:46:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:46:21.558053 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d13cbb-eece-4535-b256-40ef9174f605 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:21.656091 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893bc64e-36a7-44cd-9f75-1ddd0715bc8a None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:21.699086 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:21.735119 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300e66c3-52f9-4c40-bccf-cb017600ecbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:21.735833 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300e66c3-52f9-4c40-bccf-cb017600ecbc None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:21.736125 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300e66c3-52f9-4c40-bccf-cb017600ecbc None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:21.736409 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300e66c3-52f9-4c40-bccf-cb017600ecbc None admin] PATH_INFO: `/v3/projects/55927449396d4c4f8470544dd7a23ed9` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:21.737971 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300e66c3-52f9-4c40-bccf-cb017600ecbc None admin] RBAC: Authorizing `identity:get_project(project_id=55927449396d4c4f8470544dd7a23ed9)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:21.740796 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300e66c3-52f9-4c40-bccf-cb017600ecbc None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:21.742738 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 172/373] 10.222.0.46 () {56 vars in 1186 bytes} [Mon Oct 10 11:46:21 2022] GET /identity/v3/projects/55927449396d4c4f8470544dd7a23ed9 => generated 302 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:46:21.750130 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d13cbb-eece-4535-b256-40ef9174f605 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:21.752519 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300e66c3-52f9-4c40-bccf-cb017600ecbc None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:21.807518 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead845c3-f1b7-4774-a19e-b1277337cd57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:21.808200 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead845c3-f1b7-4774-a19e-b1277337cd57 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:21.808483 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead845c3-f1b7-4774-a19e-b1277337cd57 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:21.808756 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead845c3-f1b7-4774-a19e-b1277337cd57 None admin] PATH_INFO: `/v3/roles` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:21.809492 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead845c3-f1b7-4774-a19e-b1277337cd57 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:46:21.809797 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead845c3-f1b7-4774-a19e-b1277337cd57 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:21.811220 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead845c3-f1b7-4774-a19e-b1277337cd57 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:21.816992 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:21.833285 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 202/374] 10.222.0.46 () {56 vars in 1099 bytes} [Mon Oct 10 11:46:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:46:21.839274 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead845c3-f1b7-4774-a19e-b1277337cd57 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:21.967222 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300e66c3-52f9-4c40-bccf-cb017600ecbc None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:22.042015 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3783e7-da75-4726-9e8c-a52ff9ea4510 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:22.042817 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3783e7-da75-4726-9e8c-a52ff9ea4510 None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:22.043188 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3783e7-da75-4726-9e8c-a52ff9ea4510 None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:22.043584 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3783e7-da75-4726-9e8c-a52ff9ea4510 None admin] PATH_INFO: `/v3/groups/3ec08ca1a9da4374b4f87de87f03b1f4` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:22.047872 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3783e7-da75-4726-9e8c-a52ff9ea4510 None admin] RBAC: Authorizing `identity:get_group(group_id=3ec08ca1a9da4374b4f87de87f03b1f4)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:22.050062 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3783e7-da75-4726-9e8c-a52ff9ea4510 None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:22.052862 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 173/375] 10.222.0.46 () {56 vars in 1180 bytes} [Mon Oct 10 11:46:21 2022] GET /identity/v3/groups/3ec08ca1a9da4374b4f87de87f03b1f4 => generated 226 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:46:22.059944 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3783e7-da75-4726-9e8c-a52ff9ea4510 None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:22.061282 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead845c3-f1b7-4774-a19e-b1277337cd57 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:22.124808 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02d55fd-3c45-459e-9e62-3f41ab61cf6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:22.126401 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02d55fd-3c45-459e-9e62-3f41ab61cf6c None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:22.127725 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02d55fd-3c45-459e-9e62-3f41ab61cf6c None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:22.128261 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02d55fd-3c45-459e-9e62-3f41ab61cf6c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:22.166398 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:22.222242 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:22.231139 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02d55fd-3c45-459e-9e62-3f41ab61cf6c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:22.232594 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02d55fd-3c45-459e-9e62-3f41ab61cf6c None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:22.248696 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3783e7-da75-4726-9e8c-a52ff9ea4510 None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:22.300803 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60c7ca4-cb50-40fa-9e33-2bfb2562042f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:22.301774 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60c7ca4-cb50-40fa-9e33-2bfb2562042f None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:22.302351 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60c7ca4-cb50-40fa-9e33-2bfb2562042f None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:22.302845 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60c7ca4-cb50-40fa-9e33-2bfb2562042f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:22.303750 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60c7ca4-cb50-40fa-9e33-2bfb2562042f None admin] RBAC: Adding query filter params (group.id=3ec08ca1a9da4374b4f87de87f03b1f4, role.id=6762beebc2bf4e6f804cbbe55a475e70, scope.project.id=55927449396d4c4f8470544dd7a23ed9) {{(pid=81676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:46:22.304249 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60c7ca4-cb50-40fa-9e33-2bfb2562042f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:22.306146 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60c7ca4-cb50-40fa-9e33-2bfb2562042f None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:22.313254 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 174/376] 10.222.0.46 () {56 vars in 1376 bytes} [Mon Oct 10 11:46:22 2022] GET /identity/v3/role_assignments?group.id=3ec08ca1a9da4374b4f87de87f03b1f4&scope.project.id=55927449396d4c4f8470544dd7a23ed9&role.id=6762beebc2bf4e6f804cbbe55a475e70 => generated 260 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 11:46:22.345426 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 203/377] 10.222.0.46 () {58 vars in 1282 bytes} [Mon Oct 10 11:46:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 507 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 10 11:46:22.351160 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60c7ca4-cb50-40fa-9e33-2bfb2562042f None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:22.512368 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60c7ca4-cb50-40fa-9e33-2bfb2562042f None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:22.534015 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:22.560089 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795a3ff6-8450-4100-8eac-769e012ef3e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:22.560817 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795a3ff6-8450-4100-8eac-769e012ef3e7 None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:22.561101 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795a3ff6-8450-4100-8eac-769e012ef3e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:22.561416 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795a3ff6-8450-4100-8eac-769e012ef3e7 None admin] PATH_INFO: `/v3/projects/ec187f354d154dc98e045921aa4ca7fd` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:22.563822 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795a3ff6-8450-4100-8eac-769e012ef3e7 None admin] RBAC: Authorizing `identity:get_project(project_id=ec187f354d154dc98e045921aa4ca7fd)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:22.565292 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795a3ff6-8450-4100-8eac-769e012ef3e7 None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:22.567479 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 175/378] 10.222.0.46 () {56 vars in 1186 bytes} [Mon Oct 10 11:46:22 2022] GET /identity/v3/projects/ec187f354d154dc98e045921aa4ca7fd => generated 344 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:46:22.573114 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02d55fd-3c45-459e-9e62-3f41ab61cf6c None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:22.701469 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:22.707620 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02d55fd-3c45-459e-9e62-3f41ab61cf6c None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:22.751688 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b504a1-8e5d-4e97-89d7-95d63e25ca28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:22.752248 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b504a1-8e5d-4e97-89d7-95d63e25ca28 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:22.752531 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b504a1-8e5d-4e97-89d7-95d63e25ca28 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:22.752801 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b504a1-8e5d-4e97-89d7-95d63e25ca28 None admin] PATH_INFO: `/v3/groups/764cc3a3ef9645fbbe9183049fcbf504` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:22.758250 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b504a1-8e5d-4e97-89d7-95d63e25ca28 None admin] RBAC: Authorizing `identity:get_group(group_id=764cc3a3ef9645fbbe9183049fcbf504)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:22.759776 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b504a1-8e5d-4e97-89d7-95d63e25ca28 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:22.761730 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 204/379] 10.222.0.46 () {56 vars in 1180 bytes} [Mon Oct 10 11:46:22 2022] GET /identity/v3/groups/764cc3a3ef9645fbbe9183049fcbf504 => generated 229 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:46:22.767820 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795a3ff6-8450-4100-8eac-769e012ef3e7 None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:22.911412 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795a3ff6-8450-4100-8eac-769e012ef3e7 None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:22.958582 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741b25e2-a684-48ff-b1c2-630c49f9424f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:22.959256 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741b25e2-a684-48ff-b1c2-630c49f9424f None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:22.959545 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741b25e2-a684-48ff-b1c2-630c49f9424f None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:22.959824 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741b25e2-a684-48ff-b1c2-630c49f9424f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:22.960606 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741b25e2-a684-48ff-b1c2-630c49f9424f None admin] RBAC: Adding query filter params (group.id=764cc3a3ef9645fbbe9183049fcbf504, role.id=0d822a0674b6495fac623877a3b38025, scope.project.id=ec187f354d154dc98e045921aa4ca7fd) {{(pid=81676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:46:22.960926 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741b25e2-a684-48ff-b1c2-630c49f9424f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:22.962443 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741b25e2-a684-48ff-b1c2-630c49f9424f None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:22.969814 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 176/380] 10.222.0.46 () {56 vars in 1376 bytes} [Mon Oct 10 11:46:22 2022] GET /identity/v3/role_assignments?group.id=764cc3a3ef9645fbbe9183049fcbf504&scope.project.id=ec187f354d154dc98e045921aa4ca7fd&role.id=0d822a0674b6495fac623877a3b38025 => generated 260 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 11:46:23.327826 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c22f9f-6e8c-48ba-b5a7-27506958d038 None None] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:23.329049 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c22f9f-6e8c-48ba-b5a7-27506958d038 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:23.329049 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c22f9f-6e8c-48ba-b5a7-27506958d038 None None] PATH_INFO: `/` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:23.330587 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 205/381] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 10 11:46:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:46:23.336164 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8708182-629e-42b4-8db4-e73482a990ff None None] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:23.336453 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8708182-629e-42b4-8db4-e73482a990ff None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:23.336724 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8708182-629e-42b4-8db4-e73482a990ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:23.357058 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.auth.core [None req-f8708182-629e-42b4-8db4-e73482a990ff None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:46:23.358261 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-f8708182-629e-42b4-8db4-e73482a990ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:46:23.428090 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:23.463503 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 177/382] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 10 11:46:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 10 11:46:23.637065 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e13c6a-8d0c-408c-ae79-4e2db22e015f None None] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:23.637425 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e13c6a-8d0c-408c-ae79-4e2db22e015f None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:23.637617 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e13c6a-8d0c-408c-ae79-4e2db22e015f None None] PATH_INFO: `/` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:23.639266 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 206/383] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 10 11:46:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:46:23.644819 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8d4e8f-d4ff-4184-b819-fc36002178f0 None None] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:23.645109 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8d4e8f-d4ff-4184-b819-fc36002178f0 None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:23.645385 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8d4e8f-d4ff-4184-b819-fc36002178f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:23.664373 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.auth.core [None req-7d8d4e8f-d4ff-4184-b819-fc36002178f0 None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:46:23.665569 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-7d8d4e8f-d4ff-4184-b819-fc36002178f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:46:23.763536 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 178/384] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 10 11:46:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 10 11:46:23.818095 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e13c6a-8d0c-408c-ae79-4e2db22e015f None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:23.962135 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e13c6a-8d0c-408c-ae79-4e2db22e015f None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:24.009138 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59eea11-84fe-400b-a991-cdfb797b34bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:24.009877 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59eea11-84fe-400b-a991-cdfb797b34bc None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:24.010166 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59eea11-84fe-400b-a991-cdfb797b34bc None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:24.010444 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59eea11-84fe-400b-a991-cdfb797b34bc None admin] PATH_INFO: `/v3/roles/member` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:24.025999 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59eea11-84fe-400b-a991-cdfb797b34bc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 10 11:46:24.033702 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application [None req-d59eea11-84fe-400b-a991-cdfb797b34bc None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 10 11:46:24.033702 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:46:24.033702 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:46:24.033702 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:46:24.033702 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:46:24.033702 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:46:24.033702 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:46:24.033702 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:46:24.033702 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:46:24.033702 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:46:24.033702 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:46:24.033702 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:46:24.033702 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 10 11:46:24.033702 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 10 11:46:24.033702 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 10 11:46:24.033702 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 10 11:46:24.034612 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 10 11:46:24.034612 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 10 11:46:24.034612 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 10 11:46:24.034612 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 10 11:46:24.034612 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:46:24.034612 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:46:24.034612 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:46:24.034612 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:46:24.034612 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:46:24.034612 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:46:24.034612 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:46:24.034612 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application with Lock( Oct 10 11:46:24.034612 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:46:24.034612 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:46:24.034612 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:46:24.034612 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:46:24.035415 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:46:24.035415 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:46:24.035415 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:46:24.035415 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:46:24.035415 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 10 11:46:24.035415 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 10 11:46:24.035415 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 10 11:46:24.035415 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 10 11:46:24.035415 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 10 11:46:24.035415 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 10 11:46:24.035415 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 10 11:46:24.035415 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Oct 10 11:46:24.041396 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 207/385] 10.222.0.46 () {56 vars in 1099 bytes} [Mon Oct 10 11:46:23 2022] GET /identity/v3/roles/member => generated 84 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:46:24.043366 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8d4e8f-d4ff-4184-b819-fc36002178f0 None None] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:24.076473 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59eea11-84fe-400b-a991-cdfb797b34bc None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:24.132798 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:24.171682 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8d4e8f-d4ff-4184-b819-fc36002178f0 None None] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:24.208139 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:24.259695 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322d3640-6802-46a8-832d-272e88318078 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:24.260605 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322d3640-6802-46a8-832d-272e88318078 None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:24.261064 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322d3640-6802-46a8-832d-272e88318078 None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:24.261528 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322d3640-6802-46a8-832d-272e88318078 None admin] PATH_INFO: `/v3/roles` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:24.262683 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322d3640-6802-46a8-832d-272e88318078 None admin] RBAC: Adding query filter params (name=member) {{(pid=81676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:46:24.263219 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322d3640-6802-46a8-832d-272e88318078 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:24.266256 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322d3640-6802-46a8-832d-272e88318078 None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:24.292242 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 179/386] 10.222.0.46 () {56 vars in 1101 bytes} [Mon Oct 10 11:46:24 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:46:24.299557 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322d3640-6802-46a8-832d-272e88318078 None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:24.379841 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59eea11-84fe-400b-a991-cdfb797b34bc None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:24.445191 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c8d283-899a-4811-ac78-fe5c5ac970a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:24.446428 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c8d283-899a-4811-ac78-fe5c5ac970a5 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:24.447151 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c8d283-899a-4811-ac78-fe5c5ac970a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:24.447985 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c8d283-899a-4811-ac78-fe5c5ac970a5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:24.452126 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322d3640-6802-46a8-832d-272e88318078 None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:24.471527 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c8d283-899a-4811-ac78-fe5c5ac970a5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 10 11:46:24.482571 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application [None req-a2c8d283-899a-4811-ac78-fe5c5ac970a5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 10 11:46:24.482571 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:46:24.482571 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:46:24.482571 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:46:24.482571 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:46:24.482571 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:46:24.482571 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:46:24.482571 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:46:24.482571 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:46:24.482571 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:46:24.482571 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:46:24.482571 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:46:24.482571 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 10 11:46:24.482571 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 10 11:46:24.482571 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 10 11:46:24.482571 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 10 11:46:24.483475 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 10 11:46:24.483475 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 10 11:46:24.483475 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 10 11:46:24.483475 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 10 11:46:24.483475 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:46:24.483475 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:46:24.483475 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:46:24.483475 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:46:24.483475 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:46:24.483475 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:46:24.483475 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:46:24.483475 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application with Lock( Oct 10 11:46:24.483475 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:46:24.483475 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:46:24.483475 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:46:24.483475 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:46:24.484306 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:46:24.484306 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:46:24.484306 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:46:24.484306 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:46:24.484306 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 10 11:46:24.484306 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 10 11:46:24.484306 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 10 11:46:24.484306 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 10 11:46:24.484306 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 10 11:46:24.484306 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 10 11:46:24.484306 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 10 11:46:24.484306 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Oct 10 11:46:24.485414 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 208/387] 10.222.0.46 () {56 vars in 1099 bytes} [Mon Oct 10 11:46:24 2022] GET /identity/v3/roles/member => generated 84 bytes in 410 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:46:24.494601 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14854776-6b2b-4699-8a05-82a6e3391ad7 None None] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:24.495235 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14854776-6b2b-4699-8a05-82a6e3391ad7 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:24.495791 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14854776-6b2b-4699-8a05-82a6e3391ad7 None None] PATH_INFO: `/` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:24.498727 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 209/388] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 10 11:46:24 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:46:24.500587 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6d6622-b21d-4e82-b07c-0f6633ca0505 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:24.501124 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14854776-6b2b-4699-8a05-82a6e3391ad7 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:24.501691 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6d6622-b21d-4e82-b07c-0f6633ca0505 None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:24.502149 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6d6622-b21d-4e82-b07c-0f6633ca0505 None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:24.502598 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6d6622-b21d-4e82-b07c-0f6633ca0505 None admin] PATH_INFO: `/v3/groups/3ec08ca1a9da4374b4f87de87f03b1f4` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:24.504226 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6d6622-b21d-4e82-b07c-0f6633ca0505 None admin] RBAC: Authorizing `identity:get_group(group_id=3ec08ca1a9da4374b4f87de87f03b1f4)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:24.505930 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6d6622-b21d-4e82-b07c-0f6633ca0505 None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:24.508351 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 180/389] 10.222.0.46 () {56 vars in 1180 bytes} [Mon Oct 10 11:46:24 2022] GET /identity/v3/groups/3ec08ca1a9da4374b4f87de87f03b1f4 => generated 226 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:46:24.509945 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa0fadd-7977-4a92-8379-a01747022cb3 None None] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:24.510417 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa0fadd-7977-4a92-8379-a01747022cb3 None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:24.510892 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa0fadd-7977-4a92-8379-a01747022cb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:24.530569 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.auth.core [None req-afa0fadd-7977-4a92-8379-a01747022cb3 None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:46:24.532727 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-afa0fadd-7977-4a92-8379-a01747022cb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:46:24.573255 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:24.612513 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 181/390] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 10 11:46:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 10 11:46:24.613743 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa0fadd-7977-4a92-8379-a01747022cb3 None None] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:24.659481 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:24.699358 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14854776-6b2b-4699-8a05-82a6e3391ad7 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:24.731256 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa0fadd-7977-4a92-8379-a01747022cb3 None None] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:24.767311 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c9a675-5aee-4e98-9c8a-79474f90e6f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:24.768636 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c9a675-5aee-4e98-9c8a-79474f90e6f6 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:24.769323 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c9a675-5aee-4e98-9c8a-79474f90e6f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:24.769999 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c9a675-5aee-4e98-9c8a-79474f90e6f6 None admin] PATH_INFO: `/v3/roles` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:24.771308 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35340693-2e6d-4174-b5cb-e315f5eacf4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:24.771685 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c9a675-5aee-4e98-9c8a-79474f90e6f6 None admin] RBAC: Adding query filter params (name=member) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:46:24.772166 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35340693-2e6d-4174-b5cb-e315f5eacf4c None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:24.772543 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35340693-2e6d-4174-b5cb-e315f5eacf4c None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:24.772850 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c9a675-5aee-4e98-9c8a-79474f90e6f6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:24.773020 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35340693-2e6d-4174-b5cb-e315f5eacf4c None admin] PATH_INFO: `/v3/projects/2d0af77a941540ef9b78b89ebae1ef1a` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:24.774255 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35340693-2e6d-4174-b5cb-e315f5eacf4c None admin] RBAC: Authorizing `identity:get_project(project_id=2d0af77a941540ef9b78b89ebae1ef1a)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:24.775850 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35340693-2e6d-4174-b5cb-e315f5eacf4c None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:24.776217 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c9a675-5aee-4e98-9c8a-79474f90e6f6 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:24.777936 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 182/391] 10.222.0.46 () {56 vars in 1186 bytes} [Mon Oct 10 11:46:24 2022] GET /identity/v3/projects/2d0af77a941540ef9b78b89ebae1ef1a => generated 298 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:46:24.783849 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35340693-2e6d-4174-b5cb-e315f5eacf4c None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:24.793744 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 210/392] 10.222.0.46 () {56 vars in 1101 bytes} [Mon Oct 10 11:46:24 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 11:46:24.803709 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c9a675-5aee-4e98-9c8a-79474f90e6f6 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:24.828852 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:24.897597 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35340693-2e6d-4174-b5cb-e315f5eacf4c None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:24.935749 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcba04b7-cd4a-43cb-b1d5-07fd90ce3973 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:24.936579 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcba04b7-cd4a-43cb-b1d5-07fd90ce3973 None admin] REQUEST_METHOD: `PUT` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:24.936965 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcba04b7-cd4a-43cb-b1d5-07fd90ce3973 None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:24.937417 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcba04b7-cd4a-43cb-b1d5-07fd90ce3973 None admin] PATH_INFO: `/v3/projects/2d0af77a941540ef9b78b89ebae1ef1a/groups/3ec08ca1a9da4374b4f87de87f03b1f4/roles/b2cb2a6e818e47338ce071e6fedfc673` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:24.939602 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcba04b7-cd4a-43cb-b1d5-07fd90ce3973 None admin] RBAC: Authorizing `identity:create_grant(project_id=2d0af77a941540ef9b78b89ebae1ef1a, group_id=3ec08ca1a9da4374b4f87de87f03b1f4, role_id=b2cb2a6e818e47338ce071e6fedfc673)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:24.941268 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcba04b7-cd4a-43cb-b1d5-07fd90ce3973 None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:24.996063 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c9a675-5aee-4e98-9c8a-79474f90e6f6 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:25.043020 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 183/393] 10.222.0.46 () {58 vars in 1442 bytes} [Mon Oct 10 11:46:24 2022] PUT /identity/v3/projects/2d0af77a941540ef9b78b89ebae1ef1a/groups/3ec08ca1a9da4374b4f87de87f03b1f4/roles/b2cb2a6e818e47338ce071e6fedfc673 => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 10 11:46:25.072690 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f11b52-7e17-43de-8979-23c1475ec2e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:25.073967 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f11b52-7e17-43de-8979-23c1475ec2e5 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:25.074595 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f11b52-7e17-43de-8979-23c1475ec2e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:25.075267 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f11b52-7e17-43de-8979-23c1475ec2e5 None admin] PATH_INFO: `/v3/groups/3ec08ca1a9da4374b4f87de87f03b1f4` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:25.077505 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f11b52-7e17-43de-8979-23c1475ec2e5 None admin] RBAC: Authorizing `identity:get_group(group_id=3ec08ca1a9da4374b4f87de87f03b1f4)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:25.080943 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f11b52-7e17-43de-8979-23c1475ec2e5 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:25.084331 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 211/394] 10.222.0.46 () {56 vars in 1180 bytes} [Mon Oct 10 11:46:24 2022] GET /identity/v3/groups/3ec08ca1a9da4374b4f87de87f03b1f4 => generated 226 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:46:25.092834 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcba04b7-cd4a-43cb-b1d5-07fd90ce3973 None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:25.149804 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f11b52-7e17-43de-8979-23c1475ec2e5 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:25.174549 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:25.331346 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f11b52-7e17-43de-8979-23c1475ec2e5 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:25.337911 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:25.346479 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcba04b7-cd4a-43cb-b1d5-07fd90ce3973 None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:25.371140 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fca54c-9af6-4ae0-b00f-69ed668482f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:25.372419 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fca54c-9af6-4ae0-b00f-69ed668482f1 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:25.372906 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fca54c-9af6-4ae0-b00f-69ed668482f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:25.373405 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fca54c-9af6-4ae0-b00f-69ed668482f1 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:25.387368 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3fca54c-9af6-4ae0-b00f-69ed668482f1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 10 11:46:25.394272 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application [None req-c3fca54c-9af6-4ae0-b00f-69ed668482f1 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 10 11:46:25.394272 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:46:25.394272 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:46:25.394272 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:46:25.394272 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:46:25.394272 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:46:25.394272 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:46:25.394272 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:46:25.394272 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:46:25.394272 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:46:25.394272 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:46:25.394272 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:46:25.394272 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 10 11:46:25.394272 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 10 11:46:25.394272 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 10 11:46:25.394272 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 10 11:46:25.395238 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 10 11:46:25.395238 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 10 11:46:25.395238 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 10 11:46:25.395238 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 10 11:46:25.395238 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:46:25.395238 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:46:25.395238 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:46:25.395238 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:46:25.395238 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:46:25.395238 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:46:25.395238 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:46:25.395238 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application with Lock( Oct 10 11:46:25.395238 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:46:25.395238 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:46:25.395238 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:46:25.395238 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:46:25.396034 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:46:25.396034 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:46:25.396034 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:46:25.396034 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:46:25.396034 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 10 11:46:25.396034 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 10 11:46:25.396034 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 10 11:46:25.396034 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 10 11:46:25.396034 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 10 11:46:25.396034 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 10 11:46:25.396034 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 10 11:46:25.396034 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Oct 10 11:46:25.399138 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 212/395] 10.222.0.46 () {56 vars in 1114 bytes} [Mon Oct 10 11:46:25 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:46:25.401026 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6debf278-4a24-43f3-82c4-4d48293d6b0c None None] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:25.401656 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6debf278-4a24-43f3-82c4-4d48293d6b0c None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:25.402183 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6debf278-4a24-43f3-82c4-4d48293d6b0c None None] PATH_INFO: `/` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:25.404233 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 213/396] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 10 11:46:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:46:25.415031 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c76fe3f-1d34-43e5-91dc-18b1fce63f9c None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:25.415456 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c76fe3f-1d34-43e5-91dc-18b1fce63f9c None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:25.415834 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c76fe3f-1d34-43e5-91dc-18b1fce63f9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:25.429471 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a204e1f8-e40d-4679-9738-59ae1e2ef667 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:25.430907 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a204e1f8-e40d-4679-9738-59ae1e2ef667 None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:25.432272 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a204e1f8-e40d-4679-9738-59ae1e2ef667 None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:25.433334 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a204e1f8-e40d-4679-9738-59ae1e2ef667 None admin] PATH_INFO: `/v3/projects/55927449396d4c4f8470544dd7a23ed9` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:25.435735 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a204e1f8-e40d-4679-9738-59ae1e2ef667 None admin] RBAC: Authorizing `identity:get_project(project_id=55927449396d4c4f8470544dd7a23ed9)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:25.438207 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-2c76fe3f-1d34-43e5-91dc-18b1fce63f9c None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:46:25.441699 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-2c76fe3f-1d34-43e5-91dc-18b1fce63f9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:46:25.441872 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a204e1f8-e40d-4679-9738-59ae1e2ef667 None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:25.445238 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 184/397] 10.222.0.46 () {56 vars in 1186 bytes} [Mon Oct 10 11:46:25 2022] GET /identity/v3/projects/55927449396d4c4f8470544dd7a23ed9 => generated 302 bytes in 353 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:46:25.446977 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a204e1f8-e40d-4679-9738-59ae1e2ef667 None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:25.554624 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:25.575230 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 214/398] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 10 11:46:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 10 11:46:25.575230 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c76fe3f-1d34-43e5-91dc-18b1fce63f9c None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:25.618983 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a204e1f8-e40d-4679-9738-59ae1e2ef667 None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:25.669457 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297dcb57-b000-4b62-ab8d-dae88f37ca7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:25.670086 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297dcb57-b000-4b62-ab8d-dae88f37ca7e None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:25.670364 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297dcb57-b000-4b62-ab8d-dae88f37ca7e None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:25.670636 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297dcb57-b000-4b62-ab8d-dae88f37ca7e None admin] PATH_INFO: `/v3/roles` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:25.672979 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297dcb57-b000-4b62-ab8d-dae88f37ca7e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:46:25.673288 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297dcb57-b000-4b62-ab8d-dae88f37ca7e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:25.675039 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297dcb57-b000-4b62-ab8d-dae88f37ca7e None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:25.692107 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 185/399] 10.222.0.46 () {56 vars in 1111 bytes} [Mon Oct 10 11:46:25 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:46:25.698059 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297dcb57-b000-4b62-ab8d-dae88f37ca7e None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:25.747502 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:25.784490 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c76fe3f-1d34-43e5-91dc-18b1fce63f9c None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:25.822702 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d89b50d-95a0-461d-9129-3f570c471c65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:25.823472 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d89b50d-95a0-461d-9129-3f570c471c65 None admin] REQUEST_METHOD: `PUT` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:25.823756 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d89b50d-95a0-461d-9129-3f570c471c65 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:25.824036 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d89b50d-95a0-461d-9129-3f570c471c65 None admin] PATH_INFO: `/v3/projects/55927449396d4c4f8470544dd7a23ed9/groups/3ec08ca1a9da4374b4f87de87f03b1f4/roles/b2cb2a6e818e47338ce071e6fedfc673` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:25.826003 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d89b50d-95a0-461d-9129-3f570c471c65 None admin] RBAC: Authorizing `identity:create_grant(project_id=55927449396d4c4f8470544dd7a23ed9, group_id=3ec08ca1a9da4374b4f87de87f03b1f4, role_id=b2cb2a6e818e47338ce071e6fedfc673)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:25.827604 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d89b50d-95a0-461d-9129-3f570c471c65 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:25.885319 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297dcb57-b000-4b62-ab8d-dae88f37ca7e None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:25.934683 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 215/400] 10.222.0.46 () {58 vars in 1442 bytes} [Mon Oct 10 11:46:25 2022] PUT /identity/v3/projects/55927449396d4c4f8470544dd7a23ed9/groups/3ec08ca1a9da4374b4f87de87f03b1f4/roles/b2cb2a6e818e47338ce071e6fedfc673 => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 11:46:25.965350 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4acdf6a-5acc-48cf-98ef-122c0d0b4322 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:25.967019 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4acdf6a-5acc-48cf-98ef-122c0d0b4322 None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:25.967844 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4acdf6a-5acc-48cf-98ef-122c0d0b4322 None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:25.968647 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4acdf6a-5acc-48cf-98ef-122c0d0b4322 None admin] PATH_INFO: `/v3/groups/3ec08ca1a9da4374b4f87de87f03b1f4` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:25.971094 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4acdf6a-5acc-48cf-98ef-122c0d0b4322 None admin] RBAC: Authorizing `identity:get_group(group_id=3ec08ca1a9da4374b4f87de87f03b1f4)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:25.973851 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4acdf6a-5acc-48cf-98ef-122c0d0b4322 None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:25.977177 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 186/401] 10.222.0.46 () {56 vars in 1180 bytes} [Mon Oct 10 11:46:25 2022] GET /identity/v3/groups/3ec08ca1a9da4374b4f87de87f03b1f4 => generated 226 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 11:46:25.986058 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d89b50d-95a0-461d-9129-3f570c471c65 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:26.048445 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4acdf6a-5acc-48cf-98ef-122c0d0b4322 None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:26.155201 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:26.259345 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d89b50d-95a0-461d-9129-3f570c471c65 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:26.261443 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:26.270231 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4acdf6a-5acc-48cf-98ef-122c0d0b4322 None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:26.318722 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cabfb5e-ea13-4a3a-bfa5-f102087d1033 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:26.320060 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cabfb5e-ea13-4a3a-bfa5-f102087d1033 None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:26.320690 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cabfb5e-ea13-4a3a-bfa5-f102087d1033 None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:26.321260 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cabfb5e-ea13-4a3a-bfa5-f102087d1033 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:26.335047 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cabfb5e-ea13-4a3a-bfa5-f102087d1033 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 10 11:46:26.340355 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application [None req-3cabfb5e-ea13-4a3a-bfa5-f102087d1033 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 10 11:46:26.340355 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:46:26.340355 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:46:26.340355 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:46:26.340355 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:46:26.340355 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:46:26.340355 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:46:26.340355 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:46:26.340355 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:46:26.340355 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:46:26.340355 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:46:26.340355 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:46:26.340355 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 10 11:46:26.340355 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 10 11:46:26.340355 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 10 11:46:26.340355 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 10 11:46:26.342251 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 10 11:46:26.342251 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 10 11:46:26.342251 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 10 11:46:26.342251 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 10 11:46:26.342251 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:46:26.342251 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:46:26.342251 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:46:26.342251 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:46:26.342251 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:46:26.342251 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:46:26.342251 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:46:26.342251 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application with Lock( Oct 10 11:46:26.342251 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:46:26.342251 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:46:26.342251 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:46:26.342251 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:46:26.344691 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:46:26.344691 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:46:26.344691 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:46:26.344691 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:46:26.344691 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 10 11:46:26.344691 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 10 11:46:26.344691 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 10 11:46:26.344691 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 10 11:46:26.344691 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 10 11:46:26.344691 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 10 11:46:26.344691 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 10 11:46:26.344691 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Oct 10 11:46:26.349363 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 187/402] 10.222.0.46 () {56 vars in 1114 bytes} [Mon Oct 10 11:46:26 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 297 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 10 11:46:26.358701 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cabfb5e-ea13-4a3a-bfa5-f102087d1033 None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:26.358935 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f91dbd-e813-4982-a7ae-f30d656f6399 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:26.358935 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f91dbd-e813-4982-a7ae-f30d656f6399 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:26.358935 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f91dbd-e813-4982-a7ae-f30d656f6399 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:26.358935 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f91dbd-e813-4982-a7ae-f30d656f6399 None admin] PATH_INFO: `/v3/projects/2d0af77a941540ef9b78b89ebae1ef1a` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:26.359554 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f91dbd-e813-4982-a7ae-f30d656f6399 None admin] RBAC: Authorizing `identity:get_project(project_id=2d0af77a941540ef9b78b89ebae1ef1a)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:26.359554 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f91dbd-e813-4982-a7ae-f30d656f6399 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:26.362222 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 216/403] 10.222.0.46 () {56 vars in 1186 bytes} [Mon Oct 10 11:46:25 2022] GET /identity/v3/projects/2d0af77a941540ef9b78b89ebae1ef1a => generated 298 bytes in 377 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 11:46:26.370194 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f91dbd-e813-4982-a7ae-f30d656f6399 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:26.505854 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cabfb5e-ea13-4a3a-bfa5-f102087d1033 None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:26.565237 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3f1899-0b4d-45c3-abb7-f9bacf6e709d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:26.566425 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3f1899-0b4d-45c3-abb7-f9bacf6e709d None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:26.566917 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3f1899-0b4d-45c3-abb7-f9bacf6e709d None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:26.567389 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3f1899-0b4d-45c3-abb7-f9bacf6e709d None admin] PATH_INFO: `/v3/roles` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:26.568670 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3f1899-0b4d-45c3-abb7-f9bacf6e709d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:46:26.569344 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3f1899-0b4d-45c3-abb7-f9bacf6e709d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:26.572108 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3f1899-0b4d-45c3-abb7-f9bacf6e709d None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:26.579518 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f91dbd-e813-4982-a7ae-f30d656f6399 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:26.590227 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 188/404] 10.222.0.46 () {56 vars in 1111 bytes} [Mon Oct 10 11:46:26 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 11:46:26.592437 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b29b0a-0b8c-490a-97da-ab387de1577b None None] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:26.592889 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b29b0a-0b8c-490a-97da-ab387de1577b None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:26.593333 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b29b0a-0b8c-490a-97da-ab387de1577b None None] PATH_INFO: `/` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:26.597313 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 189/405] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 10 11:46:26 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:46:26.604895 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b29b0a-0b8c-490a-97da-ab387de1577b None None] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:26.616000 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:26.626411 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47363d68-6ac0-443c-afac-ba36a7be3142 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:26.637350 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47363d68-6ac0-443c-afac-ba36a7be3142 None admin] REQUEST_METHOD: `PUT` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:26.637350 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47363d68-6ac0-443c-afac-ba36a7be3142 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:26.637350 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47363d68-6ac0-443c-afac-ba36a7be3142 None admin] PATH_INFO: `/v3/projects/2d0af77a941540ef9b78b89ebae1ef1a/groups/3ec08ca1a9da4374b4f87de87f03b1f4/roles/6762beebc2bf4e6f804cbbe55a475e70` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:26.637350 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47363d68-6ac0-443c-afac-ba36a7be3142 None admin] RBAC: Authorizing `identity:create_grant(project_id=2d0af77a941540ef9b78b89ebae1ef1a, group_id=3ec08ca1a9da4374b4f87de87f03b1f4, role_id=6762beebc2bf4e6f804cbbe55a475e70)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:26.637350 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47363d68-6ac0-443c-afac-ba36a7be3142 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:26.655617 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:26.743867 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 217/406] 10.222.0.46 () {58 vars in 1442 bytes} [Mon Oct 10 11:46:26 2022] PUT /identity/v3/projects/2d0af77a941540ef9b78b89ebae1ef1a/groups/3ec08ca1a9da4374b4f87de87f03b1f4/roles/6762beebc2bf4e6f804cbbe55a475e70 => generated 0 bytes in 374 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 11:46:26.745630 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1355c04b-a482-4b1c-a071-8e6dc4c502ee None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:26.746141 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1355c04b-a482-4b1c-a071-8e6dc4c502ee None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:26.746616 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1355c04b-a482-4b1c-a071-8e6dc4c502ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:26.774720 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-1355c04b-a482-4b1c-a071-8e6dc4c502ee None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:46:26.776475 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-1355c04b-a482-4b1c-a071-8e6dc4c502ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:46:26.802509 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b29b0a-0b8c-490a-97da-ab387de1577b None None] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:26.869145 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ebdf0c-9794-4d30-98df-3aabdb66f426 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:26.870232 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ebdf0c-9794-4d30-98df-3aabdb66f426 None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:26.870760 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ebdf0c-9794-4d30-98df-3aabdb66f426 None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:26.871238 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ebdf0c-9794-4d30-98df-3aabdb66f426 None admin] PATH_INFO: `/v3/groups/3ec08ca1a9da4374b4f87de87f03b1f4` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:26.872183 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 218/407] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 10 11:46:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 10 11:46:26.878693 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ebdf0c-9794-4d30-98df-3aabdb66f426 None admin] RBAC: Authorizing `identity:get_group(group_id=3ec08ca1a9da4374b4f87de87f03b1f4)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:26.878693 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ebdf0c-9794-4d30-98df-3aabdb66f426 None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:26.880596 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 190/408] 10.222.0.46 () {56 vars in 1180 bytes} [Mon Oct 10 11:46:26 2022] GET /identity/v3/groups/3ec08ca1a9da4374b4f87de87f03b1f4 => generated 226 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:46:26.886925 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1355c04b-a482-4b1c-a071-8e6dc4c502ee None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:27.004157 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1355c04b-a482-4b1c-a071-8e6dc4c502ee None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:27.027237 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:27.046057 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d9c37d-030d-461e-b423-38b029ec7483 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:27.047262 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d9c37d-030d-461e-b423-38b029ec7483 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:27.047591 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d9c37d-030d-461e-b423-38b029ec7483 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:27.047875 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d9c37d-030d-461e-b423-38b029ec7483 None admin] PATH_INFO: `/v3/projects/55927449396d4c4f8470544dd7a23ed9` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:27.049065 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d9c37d-030d-461e-b423-38b029ec7483 None admin] RBAC: Authorizing `identity:get_project(project_id=55927449396d4c4f8470544dd7a23ed9)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:27.050607 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d9c37d-030d-461e-b423-38b029ec7483 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:27.052319 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 219/409] 10.222.0.46 () {56 vars in 1186 bytes} [Mon Oct 10 11:46:26 2022] GET /identity/v3/projects/55927449396d4c4f8470544dd7a23ed9 => generated 302 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 11:46:27.059150 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ebdf0c-9794-4d30-98df-3aabdb66f426 None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:27.254512 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:27.286629 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ebdf0c-9794-4d30-98df-3aabdb66f426 None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:27.349070 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d9c37d-030d-461e-b423-38b029ec7483 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:27.349741 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c94528-51d2-499a-8620-7e9ee47bf390 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:27.350966 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c94528-51d2-499a-8620-7e9ee47bf390 None admin] REQUEST_METHOD: `PUT` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:27.351411 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c94528-51d2-499a-8620-7e9ee47bf390 None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:27.351903 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c94528-51d2-499a-8620-7e9ee47bf390 None admin] PATH_INFO: `/v3/projects/55927449396d4c4f8470544dd7a23ed9/groups/3ec08ca1a9da4374b4f87de87f03b1f4/roles/6762beebc2bf4e6f804cbbe55a475e70` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:27.355727 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c94528-51d2-499a-8620-7e9ee47bf390 None admin] RBAC: Authorizing `identity:create_grant(project_id=55927449396d4c4f8470544dd7a23ed9, group_id=3ec08ca1a9da4374b4f87de87f03b1f4, role_id=6762beebc2bf4e6f804cbbe55a475e70)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:27.358330 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c94528-51d2-499a-8620-7e9ee47bf390 None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:27.439574 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 191/410] 10.222.0.46 () {58 vars in 1442 bytes} [Mon Oct 10 11:46:27 2022] PUT /identity/v3/projects/55927449396d4c4f8470544dd7a23ed9/groups/3ec08ca1a9da4374b4f87de87f03b1f4/roles/6762beebc2bf4e6f804cbbe55a475e70 => generated 0 bytes in 381 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 10 11:46:27.607382 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d9c37d-030d-461e-b423-38b029ec7483 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:27.683123 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c61e7dd-f298-463d-98ab-6d9bda4b09bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:27.684257 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c61e7dd-f298-463d-98ab-6d9bda4b09bd None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:27.684707 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c61e7dd-f298-463d-98ab-6d9bda4b09bd None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:27.685148 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c61e7dd-f298-463d-98ab-6d9bda4b09bd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:27.694470 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c145148a-8537-4c90-9420-6f2b1c884844 None None] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:27.694886 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c145148a-8537-4c90-9420-6f2b1c884844 None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:27.695275 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c145148a-8537-4c90-9420-6f2b1c884844 None None] PATH_INFO: `/` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:27.696984 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 192/411] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 10 11:46:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 10 11:46:27.702892 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9474e22-4911-4891-8a9b-7710ade946ef None None] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:27.703658 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9474e22-4911-4891-8a9b-7710ade946ef None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:27.703952 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9474e22-4911-4891-8a9b-7710ade946ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:27.715954 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c61e7dd-f298-463d-98ab-6d9bda4b09bd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 10 11:46:27.725988 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application [None req-7c61e7dd-f298-463d-98ab-6d9bda4b09bd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 10 11:46:27.725988 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:46:27.725988 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:46:27.725988 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:46:27.725988 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:46:27.725988 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:46:27.725988 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:46:27.725988 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:46:27.725988 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:46:27.725988 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:46:27.725988 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:46:27.725988 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:46:27.725988 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 10 11:46:27.725988 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 10 11:46:27.725988 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 10 11:46:27.725988 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 10 11:46:27.727998 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 10 11:46:27.727998 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 10 11:46:27.727998 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 10 11:46:27.727998 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 10 11:46:27.727998 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:46:27.727998 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:46:27.727998 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:46:27.727998 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:46:27.727998 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:46:27.727998 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:46:27.727998 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:46:27.727998 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application with Lock( Oct 10 11:46:27.727998 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:46:27.727998 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:46:27.727998 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:46:27.727998 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:46:27.729866 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:46:27.729866 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:46:27.729866 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:46:27.729866 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:46:27.729866 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 10 11:46:27.729866 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 10 11:46:27.729866 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 10 11:46:27.729866 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 10 11:46:27.729866 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 10 11:46:27.729866 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 10 11:46:27.729866 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 10 11:46:27.729866 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Oct 10 11:46:27.735432 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 220/412] 10.222.0.46 () {56 vars in 1096 bytes} [Mon Oct 10 11:46:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 387 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:46:27.745806 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.auth.core [None req-e9474e22-4911-4891-8a9b-7710ade946ef None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:46:27.747352 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-e9474e22-4911-4891-8a9b-7710ade946ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:46:27.751948 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c61e7dd-f298-463d-98ab-6d9bda4b09bd None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:27.839709 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:27.886821 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 193/413] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 10 11:46:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 10 11:46:27.927626 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c61e7dd-f298-463d-98ab-6d9bda4b09bd None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:27.987110 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5d46ac-93a0-4e63-bbae-ab6f2fc8c8df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:27.988113 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5d46ac-93a0-4e63-bbae-ab6f2fc8c8df None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:27.988597 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5d46ac-93a0-4e63-bbae-ab6f2fc8c8df None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:27.989074 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5d46ac-93a0-4e63-bbae-ab6f2fc8c8df None admin] PATH_INFO: `/v3/roles` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:27.990274 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5d46ac-93a0-4e63-bbae-ab6f2fc8c8df None admin] RBAC: Adding query filter params (name=admin) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:46:27.990822 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5d46ac-93a0-4e63-bbae-ab6f2fc8c8df None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:27.993273 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5d46ac-93a0-4e63-bbae-ab6f2fc8c8df None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:28.010120 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 221/414] 10.222.0.46 () {56 vars in 1099 bytes} [Mon Oct 10 11:46:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 11:46:28.016958 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9474e22-4911-4891-8a9b-7710ade946ef None None] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:28.183036 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5d46ac-93a0-4e63-bbae-ab6f2fc8c8df None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:28.230444 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:28.307920 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9474e22-4911-4891-8a9b-7710ade946ef None None] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:28.429937 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906b662b-dc81-4a97-9d84-74ff7798e07a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:28.431443 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906b662b-dc81-4a97-9d84-74ff7798e07a None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:28.432216 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906b662b-dc81-4a97-9d84-74ff7798e07a None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:28.432984 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906b662b-dc81-4a97-9d84-74ff7798e07a None admin] PATH_INFO: `/v3/groups/764cc3a3ef9645fbbe9183049fcbf504` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:28.439911 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906b662b-dc81-4a97-9d84-74ff7798e07a None admin] RBAC: Authorizing `identity:get_group(group_id=764cc3a3ef9645fbbe9183049fcbf504)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:28.439911 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906b662b-dc81-4a97-9d84-74ff7798e07a None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:28.441666 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 194/415] 10.222.0.46 () {56 vars in 1180 bytes} [Mon Oct 10 11:46:28 2022] GET /identity/v3/groups/764cc3a3ef9645fbbe9183049fcbf504 => generated 229 bytes in 426 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:46:28.454925 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906b662b-dc81-4a97-9d84-74ff7798e07a None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:28.496124 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5d46ac-93a0-4e63-bbae-ab6f2fc8c8df None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:28.539725 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:28.607946 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68dae1c0-25a7-4374-9e6d-fb7f6e497aa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:28.608936 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68dae1c0-25a7-4374-9e6d-fb7f6e497aa4 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:28.608936 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68dae1c0-25a7-4374-9e6d-fb7f6e497aa4 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:28.609280 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68dae1c0-25a7-4374-9e6d-fb7f6e497aa4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:28.626155 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68dae1c0-25a7-4374-9e6d-fb7f6e497aa4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 10 11:46:28.631707 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application [None req-68dae1c0-25a7-4374-9e6d-fb7f6e497aa4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 10 11:46:28.631707 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:46:28.631707 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:46:28.631707 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:46:28.631707 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:46:28.631707 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:46:28.631707 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:46:28.631707 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:46:28.631707 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:46:28.631707 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:46:28.631707 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:46:28.631707 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:46:28.631707 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 10 11:46:28.631707 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 10 11:46:28.631707 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 10 11:46:28.631707 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 10 11:46:28.633552 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 10 11:46:28.633552 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 10 11:46:28.633552 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 10 11:46:28.633552 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 10 11:46:28.633552 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:46:28.633552 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:46:28.633552 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:46:28.633552 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:46:28.633552 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:46:28.633552 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:46:28.633552 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:46:28.633552 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application with Lock( Oct 10 11:46:28.633552 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:46:28.633552 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:46:28.633552 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:46:28.633552 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:46:28.635155 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:46:28.635155 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:46:28.635155 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:46:28.635155 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:46:28.635155 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 10 11:46:28.635155 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 10 11:46:28.635155 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 10 11:46:28.635155 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 10 11:46:28.635155 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 10 11:46:28.635155 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 10 11:46:28.635155 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 10 11:46:28.635155 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Oct 10 11:46:28.635155 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 222/416] 10.222.0.46 () {56 vars in 1099 bytes} [Mon Oct 10 11:46:28 2022] GET /identity/v3/roles/member => generated 84 bytes in 451 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 10 11:46:28.642385 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68dae1c0-25a7-4374-9e6d-fb7f6e497aa4 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:28.756076 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68dae1c0-25a7-4374-9e6d-fb7f6e497aa4 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:28.768221 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906b662b-dc81-4a97-9d84-74ff7798e07a None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:28.798699 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c3cccc-1c21-481b-9f47-b1baefb1ba31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:28.798699 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c3cccc-1c21-481b-9f47-b1baefb1ba31 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:28.798699 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c3cccc-1c21-481b-9f47-b1baefb1ba31 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:28.798699 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c3cccc-1c21-481b-9f47-b1baefb1ba31 None admin] PATH_INFO: `/v3/roles` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:28.799315 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c3cccc-1c21-481b-9f47-b1baefb1ba31 None admin] RBAC: Adding query filter params (name=member) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:46:28.799315 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c3cccc-1c21-481b-9f47-b1baefb1ba31 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:28.799315 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c3cccc-1c21-481b-9f47-b1baefb1ba31 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:28.810891 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 223/417] 10.222.0.46 () {56 vars in 1101 bytes} [Mon Oct 10 11:46:28 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 11:46:28.814545 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c3cccc-1c21-481b-9f47-b1baefb1ba31 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:28.883113 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:28.926213 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c3cccc-1c21-481b-9f47-b1baefb1ba31 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:28.927910 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abce1717-cb3f-42c5-8bf6-b7c59c9475c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:28.929424 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abce1717-cb3f-42c5-8bf6-b7c59c9475c1 None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:28.930219 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abce1717-cb3f-42c5-8bf6-b7c59c9475c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:28.931013 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abce1717-cb3f-42c5-8bf6-b7c59c9475c1 None admin] PATH_INFO: `/v3/projects/ec187f354d154dc98e045921aa4ca7fd` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:28.934703 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abce1717-cb3f-42c5-8bf6-b7c59c9475c1 None admin] RBAC: Authorizing `identity:get_project(project_id=ec187f354d154dc98e045921aa4ca7fd)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:28.946705 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abce1717-cb3f-42c5-8bf6-b7c59c9475c1 None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:28.951149 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 195/418] 10.222.0.46 () {56 vars in 1186 bytes} [Mon Oct 10 11:46:28 2022] GET /identity/v3/projects/ec187f354d154dc98e045921aa4ca7fd => generated 344 bytes in 500 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 11:46:28.959535 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abce1717-cb3f-42c5-8bf6-b7c59c9475c1 None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:28.978304 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d818999-3ee6-4e91-8a40-93831d76b9c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:28.978959 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d818999-3ee6-4e91-8a40-93831d76b9c3 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:28.979246 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d818999-3ee6-4e91-8a40-93831d76b9c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:28.979572 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d818999-3ee6-4e91-8a40-93831d76b9c3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:28.994226 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:29.114540 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d818999-3ee6-4e91-8a40-93831d76b9c3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:29.115946 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d818999-3ee6-4e91-8a40-93831d76b9c3 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:29.158734 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abce1717-cb3f-42c5-8bf6-b7c59c9475c1 None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:29.176505 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:29.230636 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc7ff7a-e2a9-41e8-948f-084183fd768b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:29.231391 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc7ff7a-e2a9-41e8-948f-084183fd768b None admin] REQUEST_METHOD: `PUT` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:29.231827 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc7ff7a-e2a9-41e8-948f-084183fd768b None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:29.232188 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc7ff7a-e2a9-41e8-948f-084183fd768b None admin] PATH_INFO: `/v3/projects/ec187f354d154dc98e045921aa4ca7fd/groups/764cc3a3ef9645fbbe9183049fcbf504/roles/0d822a0674b6495fac623877a3b38025` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:29.237844 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc7ff7a-e2a9-41e8-948f-084183fd768b None admin] RBAC: Authorizing `identity:create_grant(project_id=ec187f354d154dc98e045921aa4ca7fd, group_id=764cc3a3ef9645fbbe9183049fcbf504, role_id=0d822a0674b6495fac623877a3b38025)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:29.240749 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc7ff7a-e2a9-41e8-948f-084183fd768b None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:29.309689 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 224/419] 10.222.0.46 () {58 vars in 1282 bytes} [Mon Oct 10 11:46:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 491 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Oct 10 11:46:29.311142 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d818999-3ee6-4e91-8a40-93831d76b9c3 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:29.333086 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 196/420] 10.222.0.46 () {58 vars in 1442 bytes} [Mon Oct 10 11:46:28 2022] PUT /identity/v3/projects/ec187f354d154dc98e045921aa4ca7fd/groups/764cc3a3ef9645fbbe9183049fcbf504/roles/0d822a0674b6495fac623877a3b38025 => generated 0 bytes in 374 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 11:46:29.588611 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d818999-3ee6-4e91-8a40-93831d76b9c3 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:29.628735 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:29.638172 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f297bf86-0f97-4613-a4cf-55c84a8228c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:29.638877 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f297bf86-0f97-4613-a4cf-55c84a8228c7 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:29.639187 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f297bf86-0f97-4613-a4cf-55c84a8228c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:29.639465 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f297bf86-0f97-4613-a4cf-55c84a8228c7 None admin] PATH_INFO: `/v3/projects/2d0af77a941540ef9b78b89ebae1ef1a` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:29.640707 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f297bf86-0f97-4613-a4cf-55c84a8228c7 None admin] RBAC: Authorizing `identity:get_project(project_id=2d0af77a941540ef9b78b89ebae1ef1a)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:29.642172 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f297bf86-0f97-4613-a4cf-55c84a8228c7 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:29.644226 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 225/421] 10.222.0.46 () {56 vars in 1186 bytes} [Mon Oct 10 11:46:29 2022] GET /identity/v3/projects/2d0af77a941540ef9b78b89ebae1ef1a => generated 298 bytes in 333 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:46:29.650110 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc7ff7a-e2a9-41e8-948f-084183fd768b None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:29.799881 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:29.841831 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc7ff7a-e2a9-41e8-948f-084183fd768b None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:29.892451 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08eae730-3496-4d19-afc5-31b5a7d232ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:29.893129 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08eae730-3496-4d19-afc5-31b5a7d232ca None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:29.893415 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08eae730-3496-4d19-afc5-31b5a7d232ca None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:29.893697 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08eae730-3496-4d19-afc5-31b5a7d232ca None admin] PATH_INFO: `/v3/groups/3ec08ca1a9da4374b4f87de87f03b1f4` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:29.899639 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08eae730-3496-4d19-afc5-31b5a7d232ca None admin] RBAC: Authorizing `identity:get_group(group_id=3ec08ca1a9da4374b4f87de87f03b1f4)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:29.901154 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08eae730-3496-4d19-afc5-31b5a7d232ca None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:29.902961 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 197/422] 10.222.0.46 () {56 vars in 1180 bytes} [Mon Oct 10 11:46:29 2022] GET /identity/v3/groups/3ec08ca1a9da4374b4f87de87f03b1f4 => generated 226 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 11:46:29.909490 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f297bf86-0f97-4613-a4cf-55c84a8228c7 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:30.058540 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f297bf86-0f97-4613-a4cf-55c84a8228c7 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:30.116646 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28cb3b4-1ec6-40cb-a4a6-bbc61f671c7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:30.117328 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28cb3b4-1ec6-40cb-a4a6-bbc61f671c7a None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:30.117611 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28cb3b4-1ec6-40cb-a4a6-bbc61f671c7a None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:30.117888 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28cb3b4-1ec6-40cb-a4a6-bbc61f671c7a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:30.118661 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28cb3b4-1ec6-40cb-a4a6-bbc61f671c7a None admin] RBAC: Adding query filter params (group.id=3ec08ca1a9da4374b4f87de87f03b1f4, role.id=b2cb2a6e818e47338ce071e6fedfc673, scope.project.id=2d0af77a941540ef9b78b89ebae1ef1a) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:46:30.119009 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28cb3b4-1ec6-40cb-a4a6-bbc61f671c7a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:30.120544 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28cb3b4-1ec6-40cb-a4a6-bbc61f671c7a None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:30.131878 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d137cf9-5342-4ee6-ae08-5519702e637b None None] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:30.132364 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d137cf9-5342-4ee6-ae08-5519702e637b None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:30.132850 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d137cf9-5342-4ee6-ae08-5519702e637b None None] PATH_INFO: `/` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:30.135620 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 198/423] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 10 11:46:30 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:46:30.141497 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 226/424] 10.222.0.46 () {56 vars in 1376 bytes} [Mon Oct 10 11:46:29 2022] GET /identity/v3/role_assignments?group.id=3ec08ca1a9da4374b4f87de87f03b1f4&scope.project.id=2d0af77a941540ef9b78b89ebae1ef1a&role.id=b2cb2a6e818e47338ce071e6fedfc673 => generated 611 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 11:46:30.145759 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d61e95d-3dd6-4fdd-a675-54e4383e37d8 None None] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:30.146241 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d61e95d-3dd6-4fdd-a675-54e4383e37d8 None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:30.146757 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d61e95d-3dd6-4fdd-a675-54e4383e37d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:30.180424 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.auth.core [None req-1d61e95d-3dd6-4fdd-a675-54e4383e37d8 None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:46:30.182701 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-1d61e95d-3dd6-4fdd-a675-54e4383e37d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:46:30.228109 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d540b3-1bd7-430b-8350-d2520e1862af None None] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:30.228555 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d540b3-1bd7-430b-8350-d2520e1862af None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:30.228993 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d540b3-1bd7-430b-8350-d2520e1862af None None] PATH_INFO: `/` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:30.231660 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 227/425] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 10 11:46:30 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 10 11:46:30.240591 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62124f7f-1762-41d5-9ad8-08bfe3ee75c2 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:30.240994 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62124f7f-1762-41d5-9ad8-08bfe3ee75c2 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:30.241395 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62124f7f-1762-41d5-9ad8-08bfe3ee75c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:30.271717 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-62124f7f-1762-41d5-9ad8-08bfe3ee75c2 None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:46:30.273594 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-62124f7f-1762-41d5-9ad8-08bfe3ee75c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:46:30.338207 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 199/426] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 10 11:46:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 10 11:46:30.439117 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:30.459713 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 228/427] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 10 11:46:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 10 11:46:30.559046 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaab62dd-06a9-4c72-ada8-018c9ca731c3 None None] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:30.559378 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaab62dd-06a9-4c72-ada8-018c9ca731c3 None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:30.559617 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaab62dd-06a9-4c72-ada8-018c9ca731c3 None None] PATH_INFO: `/` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:30.561450 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 200/428] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 10 11:46:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 10 11:46:30.569218 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b92e83e-4fb7-407c-8cce-c716473eeef1 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:30.569704 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b92e83e-4fb7-407c-8cce-c716473eeef1 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:30.570166 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b92e83e-4fb7-407c-8cce-c716473eeef1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:30.604365 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-3b92e83e-4fb7-407c-8cce-c716473eeef1 None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:46:30.606463 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-3b92e83e-4fb7-407c-8cce-c716473eeef1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:46:30.752517 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 229/429] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 10 11:46:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 10 11:46:30.776006 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaab62dd-06a9-4c72-ada8-018c9ca731c3 None None] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:30.887868 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:30.939467 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaab62dd-06a9-4c72-ada8-018c9ca731c3 None None] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:30.990684 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0ff434-8f17-4497-bd88-51bf4be90d53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:30.991586 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0ff434-8f17-4497-bd88-51bf4be90d53 None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:30.992044 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0ff434-8f17-4497-bd88-51bf4be90d53 None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:30.992463 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0ff434-8f17-4497-bd88-51bf4be90d53 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:31.008721 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0ff434-8f17-4497-bd88-51bf4be90d53 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 10 11:46:31.014551 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application [None req-2f0ff434-8f17-4497-bd88-51bf4be90d53 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 10 11:46:31.014551 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:46:31.014551 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:46:31.014551 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:46:31.014551 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:46:31.014551 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:46:31.014551 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:46:31.014551 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:46:31.014551 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:46:31.014551 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:46:31.014551 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:46:31.014551 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:46:31.014551 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 10 11:46:31.014551 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 10 11:46:31.014551 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 10 11:46:31.014551 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 10 11:46:31.016233 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 10 11:46:31.016233 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 10 11:46:31.016233 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 10 11:46:31.016233 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 10 11:46:31.016233 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:46:31.016233 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:46:31.016233 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:46:31.016233 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:46:31.016233 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:46:31.016233 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:46:31.016233 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:46:31.016233 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application with Lock( Oct 10 11:46:31.016233 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:46:31.016233 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:46:31.016233 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:46:31.016233 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:46:31.017196 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:46:31.017196 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:46:31.017196 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:46:31.017196 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:46:31.017196 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 10 11:46:31.017196 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 10 11:46:31.017196 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 10 11:46:31.017196 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 10 11:46:31.017196 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 10 11:46:31.017196 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 10 11:46:31.017196 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 10 11:46:31.017196 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Oct 10 11:46:31.019646 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 201/430] 10.222.0.46 () {56 vars in 1099 bytes} [Mon Oct 10 11:46:30 2022] GET /identity/v3/roles/member => generated 84 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:46:31.024414 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b92e83e-4fb7-407c-8cce-c716473eeef1 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:31.027388 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0ff434-8f17-4497-bd88-51bf4be90d53 None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:31.172088 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0ff434-8f17-4497-bd88-51bf4be90d53 None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:31.207765 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:31.217433 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083d9760-20d7-493a-9c49-ce8fba8f8ce2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:31.225182 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083d9760-20d7-493a-9c49-ce8fba8f8ce2 None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:31.225182 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083d9760-20d7-493a-9c49-ce8fba8f8ce2 None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:31.225182 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083d9760-20d7-493a-9c49-ce8fba8f8ce2 None admin] PATH_INFO: `/v3/roles` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:31.225182 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083d9760-20d7-493a-9c49-ce8fba8f8ce2 None admin] RBAC: Adding query filter params (name=member) {{(pid=81676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:46:31.225182 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083d9760-20d7-493a-9c49-ce8fba8f8ce2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:31.225182 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083d9760-20d7-493a-9c49-ce8fba8f8ce2 None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:31.225590 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b92e83e-4fb7-407c-8cce-c716473eeef1 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:31.233635 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 202/431] 10.222.0.46 () {56 vars in 1101 bytes} [Mon Oct 10 11:46:31 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 11:46:31.240228 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083d9760-20d7-493a-9c49-ce8fba8f8ce2 None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:31.269597 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:31.287780 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ea5db1-bd98-4f87-b22a-ace1b56240d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:31.288898 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ea5db1-bd98-4f87-b22a-ace1b56240d2 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:31.289397 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ea5db1-bd98-4f87-b22a-ace1b56240d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:31.290085 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ea5db1-bd98-4f87-b22a-ace1b56240d2 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:31.308218 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ea5db1-bd98-4f87-b22a-ace1b56240d2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 10 11:46:31.318233 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application [None req-32ea5db1-bd98-4f87-b22a-ace1b56240d2 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 10 11:46:31.318233 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:46:31.318233 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:46:31.318233 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:46:31.318233 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:46:31.318233 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:46:31.318233 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:46:31.318233 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:46:31.318233 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:46:31.318233 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:46:31.318233 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:46:31.318233 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:46:31.318233 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 10 11:46:31.318233 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 10 11:46:31.318233 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 10 11:46:31.318233 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 10 11:46:31.319255 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 10 11:46:31.319255 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 10 11:46:31.319255 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 10 11:46:31.319255 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 10 11:46:31.319255 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:46:31.319255 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:46:31.319255 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:46:31.319255 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:46:31.319255 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:46:31.319255 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:46:31.319255 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:46:31.319255 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application with Lock( Oct 10 11:46:31.319255 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:46:31.319255 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:46:31.319255 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:46:31.319255 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:46:31.320064 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:46:31.320064 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:46:31.320064 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:46:31.320064 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:46:31.320064 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 10 11:46:31.320064 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 10 11:46:31.320064 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 10 11:46:31.320064 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 10 11:46:31.320064 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 10 11:46:31.320064 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 10 11:46:31.320064 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 10 11:46:31.320064 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Oct 10 11:46:31.320665 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 230/432] 10.222.0.46 () {56 vars in 1114 bytes} [Mon Oct 10 11:46:31 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 297 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:46:31.321673 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ea5db1-bd98-4f87-b22a-ace1b56240d2 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:31.394949 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083d9760-20d7-493a-9c49-ce8fba8f8ce2 None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:31.453778 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d49661-102c-46c6-815b-574c33b09a87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:31.454992 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d49661-102c-46c6-815b-574c33b09a87 None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:31.455355 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d49661-102c-46c6-815b-574c33b09a87 None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:31.455792 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d49661-102c-46c6-815b-574c33b09a87 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:31.494918 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ea5db1-bd98-4f87-b22a-ace1b56240d2 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:31.553929 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:31.554567 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375c5fa7-aa65-47ed-afe2-d2a5ec671c0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:31.556416 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375c5fa7-aa65-47ed-afe2-d2a5ec671c0f None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:31.556841 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375c5fa7-aa65-47ed-afe2-d2a5ec671c0f None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:31.557291 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375c5fa7-aa65-47ed-afe2-d2a5ec671c0f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:31.569653 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d49661-102c-46c6-815b-574c33b09a87 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:31.572114 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d49661-102c-46c6-815b-574c33b09a87 None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:31.576059 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375c5fa7-aa65-47ed-afe2-d2a5ec671c0f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 10 11:46:31.584577 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application [None req-375c5fa7-aa65-47ed-afe2-d2a5ec671c0f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 10 11:46:31.584577 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:46:31.584577 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:46:31.584577 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:46:31.584577 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:46:31.584577 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:46:31.584577 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:46:31.584577 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:46:31.584577 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:46:31.584577 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:46:31.584577 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:46:31.584577 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:46:31.584577 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 10 11:46:31.584577 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 10 11:46:31.584577 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 10 11:46:31.584577 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 10 11:46:31.585549 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 10 11:46:31.585549 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 10 11:46:31.585549 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 10 11:46:31.585549 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 10 11:46:31.585549 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:46:31.585549 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:46:31.585549 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:46:31.585549 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:46:31.585549 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:46:31.585549 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:46:31.585549 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:46:31.585549 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application with Lock( Oct 10 11:46:31.585549 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:46:31.585549 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:46:31.585549 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:46:31.585549 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:46:31.586370 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:46:31.586370 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:46:31.586370 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:46:31.586370 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:46:31.586370 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 10 11:46:31.586370 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 10 11:46:31.586370 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 10 11:46:31.586370 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 10 11:46:31.586370 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 10 11:46:31.586370 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 10 11:46:31.586370 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 10 11:46:31.586370 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Oct 10 11:46:31.586370 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 231/433] 10.222.0.46 () {56 vars in 1114 bytes} [Mon Oct 10 11:46:31 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:46:31.587650 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375c5fa7-aa65-47ed-afe2-d2a5ec671c0f None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:31.657047 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:31.720024 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 203/434] 10.222.0.46 () {58 vars in 1282 bytes} [Mon Oct 10 11:46:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 480 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 10 11:46:31.720795 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d49661-102c-46c6-815b-574c33b09a87 None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:31.735933 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375c5fa7-aa65-47ed-afe2-d2a5ec671c0f None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:31.778305 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94701d04-255b-4629-939b-5aaa54e175c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:31.779350 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94701d04-255b-4629-939b-5aaa54e175c4 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:31.779350 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94701d04-255b-4629-939b-5aaa54e175c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:31.779613 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94701d04-255b-4629-939b-5aaa54e175c4 None admin] PATH_INFO: `/v3/roles` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:31.780147 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94701d04-255b-4629-939b-5aaa54e175c4 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:46:31.780461 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94701d04-255b-4629-939b-5aaa54e175c4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:31.781901 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94701d04-255b-4629-939b-5aaa54e175c4 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:31.790744 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 232/435] 10.222.0.46 () {56 vars in 1111 bytes} [Mon Oct 10 11:46:31 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:46:31.791374 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94701d04-255b-4629-939b-5aaa54e175c4 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:31.836773 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:31.848340 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d49661-102c-46c6-815b-574c33b09a87 None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:31.891802 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c67cf01-3f42-47af-9013-c2c5525591e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:31.892426 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c67cf01-3f42-47af-9013-c2c5525591e5 None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:31.892701 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c67cf01-3f42-47af-9013-c2c5525591e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:31.892970 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c67cf01-3f42-47af-9013-c2c5525591e5 None admin] PATH_INFO: `/v3/roles` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:31.893651 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c67cf01-3f42-47af-9013-c2c5525591e5 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:46:31.893951 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c67cf01-3f42-47af-9013-c2c5525591e5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:31.895393 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c67cf01-3f42-47af-9013-c2c5525591e5 None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:31.905338 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 204/436] 10.222.0.46 () {56 vars in 1111 bytes} [Mon Oct 10 11:46:31 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:46:31.906009 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c67cf01-3f42-47af-9013-c2c5525591e5 None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:31.944051 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94701d04-255b-4629-939b-5aaa54e175c4 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:31.948615 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:32.004647 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae50fd3-a40d-4185-8d25-236d714d499a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:32.005639 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae50fd3-a40d-4185-8d25-236d714d499a None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:32.006123 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae50fd3-a40d-4185-8d25-236d714d499a None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:32.006589 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae50fd3-a40d-4185-8d25-236d714d499a None admin] PATH_INFO: `/v3/projects/55927449396d4c4f8470544dd7a23ed9` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:32.008612 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae50fd3-a40d-4185-8d25-236d714d499a None admin] RBAC: Authorizing `identity:get_project(project_id=55927449396d4c4f8470544dd7a23ed9)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:32.011391 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae50fd3-a40d-4185-8d25-236d714d499a None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:32.014483 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 233/437] 10.222.0.46 () {56 vars in 1186 bytes} [Mon Oct 10 11:46:31 2022] GET /identity/v3/projects/55927449396d4c4f8470544dd7a23ed9 => generated 302 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:46:32.015955 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae50fd3-a40d-4185-8d25-236d714d499a None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:32.044563 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c67cf01-3f42-47af-9013-c2c5525591e5 None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:32.094529 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7337d30-a6d2-4490-8cb3-124623e7a2aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:32.095453 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7337d30-a6d2-4490-8cb3-124623e7a2aa None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:32.095453 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7337d30-a6d2-4490-8cb3-124623e7a2aa None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:32.095709 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7337d30-a6d2-4490-8cb3-124623e7a2aa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:32.127991 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:32.153252 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae50fd3-a40d-4185-8d25-236d714d499a None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:32.173896 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7337d30-a6d2-4490-8cb3-124623e7a2aa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:32.175388 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7337d30-a6d2-4490-8cb3-124623e7a2aa None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:32.190383 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0269558f-2bf9-4623-b9e4-8ae87b1024e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:32.191027 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0269558f-2bf9-4623-b9e4-8ae87b1024e0 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:32.191309 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0269558f-2bf9-4623-b9e4-8ae87b1024e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:32.191580 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0269558f-2bf9-4623-b9e4-8ae87b1024e0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:32.235953 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:32.264973 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0269558f-2bf9-4623-b9e4-8ae87b1024e0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:32.266365 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0269558f-2bf9-4623-b9e4-8ae87b1024e0 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:32.281647 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 205/438] 10.222.0.46 () {58 vars in 1282 bytes} [Mon Oct 10 11:46:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 10 11:46:32.282500 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7337d30-a6d2-4490-8cb3-124623e7a2aa None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:32.340154 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:32.377536 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 234/439] 10.222.0.46 () {58 vars in 1282 bytes} [Mon Oct 10 11:46:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 10 11:46:32.379012 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf380cd1-e9ea-45d2-a343-f80ffa9b204c None None] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:32.379313 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf380cd1-e9ea-45d2-a343-f80ffa9b204c None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:32.379557 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf380cd1-e9ea-45d2-a343-f80ffa9b204c None None] PATH_INFO: `/` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:32.381072 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 235/440] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 10 11:46:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:46:32.385956 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf380cd1-e9ea-45d2-a343-f80ffa9b204c None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:32.393176 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7337d30-a6d2-4490-8cb3-124623e7a2aa None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:32.431885 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144ce833-5571-428f-83b5-0fc370b6e155 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:32.432921 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144ce833-5571-428f-83b5-0fc370b6e155 None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:32.433325 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144ce833-5571-428f-83b5-0fc370b6e155 None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:32.433720 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144ce833-5571-428f-83b5-0fc370b6e155 None admin] PATH_INFO: `/v3/groups/3ec08ca1a9da4374b4f87de87f03b1f4` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:32.435155 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144ce833-5571-428f-83b5-0fc370b6e155 None admin] RBAC: Authorizing `identity:get_group(group_id=3ec08ca1a9da4374b4f87de87f03b1f4)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:32.437628 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144ce833-5571-428f-83b5-0fc370b6e155 None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:32.439706 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 206/441] 10.222.0.46 () {56 vars in 1180 bytes} [Mon Oct 10 11:46:32 2022] GET /identity/v3/groups/3ec08ca1a9da4374b4f87de87f03b1f4 => generated 226 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:46:32.442522 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144ce833-5571-428f-83b5-0fc370b6e155 None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:32.465470 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:32.500083 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf380cd1-e9ea-45d2-a343-f80ffa9b204c None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:32.536715 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69806871-8a7e-4d9f-a529-c590a2bcc254 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:32.537237 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69806871-8a7e-4d9f-a529-c590a2bcc254 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:32.537521 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69806871-8a7e-4d9f-a529-c590a2bcc254 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:32.537794 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69806871-8a7e-4d9f-a529-c590a2bcc254 None admin] PATH_INFO: `/v3/projects/2d0af77a941540ef9b78b89ebae1ef1a` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:32.539016 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69806871-8a7e-4d9f-a529-c590a2bcc254 None admin] RBAC: Authorizing `identity:get_project(project_id=2d0af77a941540ef9b78b89ebae1ef1a)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:32.541077 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69806871-8a7e-4d9f-a529-c590a2bcc254 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:32.548466 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 236/442] 10.222.0.46 () {56 vars in 1186 bytes} [Mon Oct 10 11:46:32 2022] GET /identity/v3/projects/2d0af77a941540ef9b78b89ebae1ef1a => generated 298 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:46:32.548466 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3c4335-20cc-40fb-9f57-2b326b1e89c8 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:32.548466 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3c4335-20cc-40fb-9f57-2b326b1e89c8 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:32.548466 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3c4335-20cc-40fb-9f57-2b326b1e89c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:32.563636 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-2f3c4335-20cc-40fb-9f57-2b326b1e89c8 None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:46:32.564835 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-2f3c4335-20cc-40fb-9f57-2b326b1e89c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:46:32.565335 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144ce833-5571-428f-83b5-0fc370b6e155 None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:32.588866 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:32.605731 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7742503a-9708-42bb-9e9d-596c4cde9478 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:32.606765 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7742503a-9708-42bb-9e9d-596c4cde9478 None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:32.607106 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7742503a-9708-42bb-9e9d-596c4cde9478 None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:32.607477 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7742503a-9708-42bb-9e9d-596c4cde9478 None admin] PATH_INFO: `/v3/projects/55927449396d4c4f8470544dd7a23ed9` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:32.608819 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7742503a-9708-42bb-9e9d-596c4cde9478 None admin] RBAC: Authorizing `identity:get_project(project_id=55927449396d4c4f8470544dd7a23ed9)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:32.611289 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7742503a-9708-42bb-9e9d-596c4cde9478 None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:32.613088 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 207/443] 10.222.0.46 () {56 vars in 1186 bytes} [Mon Oct 10 11:46:32 2022] GET /identity/v3/projects/55927449396d4c4f8470544dd7a23ed9 => generated 302 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:46:32.620579 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7742503a-9708-42bb-9e9d-596c4cde9478 None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:32.642823 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 237/444] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 10 11:46:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 10 11:46:32.643767 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3c4335-20cc-40fb-9f57-2b326b1e89c8 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:32.750536 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7742503a-9708-42bb-9e9d-596c4cde9478 None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:32.758770 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:32.793227 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6f3e06-9499-40ef-b1fe-2424b0d648f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:32.793734 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6f3e06-9499-40ef-b1fe-2424b0d648f5 None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:32.794021 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6f3e06-9499-40ef-b1fe-2424b0d648f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:32.794286 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6f3e06-9499-40ef-b1fe-2424b0d648f5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:32.795067 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6f3e06-9499-40ef-b1fe-2424b0d648f5 None admin] RBAC: Adding query filter params (group.id=3ec08ca1a9da4374b4f87de87f03b1f4, role.id=b2cb2a6e818e47338ce071e6fedfc673, scope.project.id=55927449396d4c4f8470544dd7a23ed9) {{(pid=81676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:46:32.795374 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6f3e06-9499-40ef-b1fe-2424b0d648f5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:32.797492 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6f3e06-9499-40ef-b1fe-2424b0d648f5 None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:32.803208 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3c4335-20cc-40fb-9f57-2b326b1e89c8 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:32.814907 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 208/445] 10.222.0.46 () {56 vars in 1376 bytes} [Mon Oct 10 11:46:32 2022] GET /identity/v3/role_assignments?group.id=3ec08ca1a9da4374b4f87de87f03b1f4&scope.project.id=55927449396d4c4f8470544dd7a23ed9&role.id=b2cb2a6e818e47338ce071e6fedfc673 => generated 611 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:46:32.814907 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6f3e06-9499-40ef-b1fe-2424b0d648f5 None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:32.879037 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ff1639-071a-408a-9f64-a7d7cadb4bca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:32.880392 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ff1639-071a-408a-9f64-a7d7cadb4bca None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:32.880954 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ff1639-071a-408a-9f64-a7d7cadb4bca None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:32.881516 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ff1639-071a-408a-9f64-a7d7cadb4bca None admin] PATH_INFO: `/v3/groups/3ec08ca1a9da4374b4f87de87f03b1f4` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:32.883690 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ff1639-071a-408a-9f64-a7d7cadb4bca None admin] RBAC: Authorizing `identity:get_group(group_id=3ec08ca1a9da4374b4f87de87f03b1f4)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:32.886389 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ff1639-071a-408a-9f64-a7d7cadb4bca None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:32.889651 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 238/446] 10.222.0.46 () {56 vars in 1180 bytes} [Mon Oct 10 11:46:32 2022] GET /identity/v3/groups/3ec08ca1a9da4374b4f87de87f03b1f4 => generated 226 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:46:32.898884 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ff1639-071a-408a-9f64-a7d7cadb4bca None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:32.921790 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:32.988045 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6f3e06-9499-40ef-b1fe-2424b0d648f5 None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:33.049497 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6cb99ca-b041-45bf-a7ab-193d09a9649f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:33.050403 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cb99ca-b041-45bf-a7ab-193d09a9649f None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:33.051463 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cb99ca-b041-45bf-a7ab-193d09a9649f None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:33.051747 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cb99ca-b041-45bf-a7ab-193d09a9649f None admin] PATH_INFO: `/v3/groups/3ec08ca1a9da4374b4f87de87f03b1f4` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:33.053023 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6cb99ca-b041-45bf-a7ab-193d09a9649f None admin] RBAC: Authorizing `identity:get_group(group_id=3ec08ca1a9da4374b4f87de87f03b1f4)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:33.054489 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6cb99ca-b041-45bf-a7ab-193d09a9649f None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:33.056407 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 209/447] 10.222.0.46 () {56 vars in 1180 bytes} [Mon Oct 10 11:46:32 2022] GET /identity/v3/groups/3ec08ca1a9da4374b4f87de87f03b1f4 => generated 226 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:46:33.056969 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6cb99ca-b041-45bf-a7ab-193d09a9649f None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:33.061009 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ff1639-071a-408a-9f64-a7d7cadb4bca None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:33.111982 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7f904e-0d25-4846-a6aa-31f7cd21a2e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:33.112936 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7f904e-0d25-4846-a6aa-31f7cd21a2e4 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:33.113303 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7f904e-0d25-4846-a6aa-31f7cd21a2e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:33.113664 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7f904e-0d25-4846-a6aa-31f7cd21a2e4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:33.114988 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7f904e-0d25-4846-a6aa-31f7cd21a2e4 None admin] RBAC: Adding query filter params (group.id=3ec08ca1a9da4374b4f87de87f03b1f4, role.id=6762beebc2bf4e6f804cbbe55a475e70, scope.project.id=2d0af77a941540ef9b78b89ebae1ef1a) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:46:33.115502 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7f904e-0d25-4846-a6aa-31f7cd21a2e4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:33.117874 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7f904e-0d25-4846-a6aa-31f7cd21a2e4 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:33.134706 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 239/448] 10.222.0.46 () {56 vars in 1376 bytes} [Mon Oct 10 11:46:32 2022] GET /identity/v3/role_assignments?group.id=3ec08ca1a9da4374b4f87de87f03b1f4&scope.project.id=2d0af77a941540ef9b78b89ebae1ef1a&role.id=6762beebc2bf4e6f804cbbe55a475e70 => generated 611 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:46:33.134706 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7f904e-0d25-4846-a6aa-31f7cd21a2e4 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:33.134706 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:33.208370 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6cb99ca-b041-45bf-a7ab-193d09a9649f None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:33.236652 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:33.255309 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e11f738-c9dc-4fd4-9368-351e24901eb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:33.256274 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e11f738-c9dc-4fd4-9368-351e24901eb9 None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:33.256711 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e11f738-c9dc-4fd4-9368-351e24901eb9 None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:33.257137 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e11f738-c9dc-4fd4-9368-351e24901eb9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:33.269061 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7f904e-0d25-4846-a6aa-31f7cd21a2e4 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:33.275646 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e11f738-c9dc-4fd4-9368-351e24901eb9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 10 11:46:33.288339 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application [None req-3e11f738-c9dc-4fd4-9368-351e24901eb9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 10 11:46:33.288339 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:46:33.288339 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:46:33.288339 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:46:33.288339 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:46:33.288339 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:46:33.288339 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:46:33.288339 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:46:33.288339 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:46:33.288339 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:46:33.288339 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:46:33.288339 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:46:33.288339 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 10 11:46:33.288339 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 10 11:46:33.288339 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 10 11:46:33.288339 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 10 11:46:33.289302 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 10 11:46:33.289302 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 10 11:46:33.289302 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 10 11:46:33.289302 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 10 11:46:33.289302 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:46:33.289302 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:46:33.289302 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:46:33.289302 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:46:33.289302 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:46:33.289302 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:46:33.289302 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:46:33.289302 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application with Lock( Oct 10 11:46:33.289302 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:46:33.289302 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:46:33.289302 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:46:33.289302 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:46:33.290066 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:46:33.290066 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:46:33.290066 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:46:33.290066 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:46:33.290066 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 10 11:46:33.290066 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 10 11:46:33.290066 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 10 11:46:33.290066 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 10 11:46:33.290066 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 10 11:46:33.290066 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 10 11:46:33.290066 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 10 11:46:33.290066 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Oct 10 11:46:33.290662 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 210/449] 10.222.0.46 () {56 vars in 1096 bytes} [Mon Oct 10 11:46:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:46:33.297166 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e11f738-c9dc-4fd4-9368-351e24901eb9 None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:33.325906 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea48395-dd22-4e2e-acc0-ce87a1052c45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:33.326873 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea48395-dd22-4e2e-acc0-ce87a1052c45 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:33.327334 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea48395-dd22-4e2e-acc0-ce87a1052c45 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:33.327878 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea48395-dd22-4e2e-acc0-ce87a1052c45 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:33.329112 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea48395-dd22-4e2e-acc0-ce87a1052c45 None admin] RBAC: Adding query filter params (group.id=3ec08ca1a9da4374b4f87de87f03b1f4, role.id=6762beebc2bf4e6f804cbbe55a475e70, scope.project.id=55927449396d4c4f8470544dd7a23ed9) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:46:33.329581 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea48395-dd22-4e2e-acc0-ce87a1052c45 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:33.332071 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea48395-dd22-4e2e-acc0-ce87a1052c45 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:33.341674 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 240/450] 10.222.0.46 () {56 vars in 1376 bytes} [Mon Oct 10 11:46:33 2022] GET /identity/v3/role_assignments?group.id=3ec08ca1a9da4374b4f87de87f03b1f4&scope.project.id=55927449396d4c4f8470544dd7a23ed9&role.id=6762beebc2bf4e6f804cbbe55a475e70 => generated 611 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:46:33.441095 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e11f738-c9dc-4fd4-9368-351e24901eb9 None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:33.493278 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90157b1-1fbc-43b9-bbc1-21370f8fd8d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:33.493924 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90157b1-1fbc-43b9-bbc1-21370f8fd8d5 None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:33.494263 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90157b1-1fbc-43b9-bbc1-21370f8fd8d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:33.494533 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90157b1-1fbc-43b9-bbc1-21370f8fd8d5 None admin] PATH_INFO: `/v3/roles` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:33.495208 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90157b1-1fbc-43b9-bbc1-21370f8fd8d5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:46:33.495514 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90157b1-1fbc-43b9-bbc1-21370f8fd8d5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:33.497213 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90157b1-1fbc-43b9-bbc1-21370f8fd8d5 None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:33.508005 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 211/451] 10.222.0.46 () {56 vars in 1099 bytes} [Mon Oct 10 11:46:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 11:46:33.513947 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea48395-dd22-4e2e-acc0-ce87a1052c45 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:33.614810 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:33.662257 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea48395-dd22-4e2e-acc0-ce87a1052c45 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:33.702020 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4090a1-83dc-462d-ba2a-d9d9ba87a836 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:33.703281 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4090a1-83dc-462d-ba2a-d9d9ba87a836 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:33.703621 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4090a1-83dc-462d-ba2a-d9d9ba87a836 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:33.703954 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4090a1-83dc-462d-ba2a-d9d9ba87a836 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:33.784053 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4090a1-83dc-462d-ba2a-d9d9ba87a836 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:33.785525 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4090a1-83dc-462d-ba2a-d9d9ba87a836 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:33.855738 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:33.897971 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 241/452] 10.222.0.46 () {58 vars in 1282 bytes} [Mon Oct 10 11:46:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Oct 10 11:46:33.903457 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90157b1-1fbc-43b9-bbc1-21370f8fd8d5 None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:33.961810 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:34.020372 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90157b1-1fbc-43b9-bbc1-21370f8fd8d5 None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:34.053734 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56367d6-f975-4053-b16f-6050aee1d664 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:34.054364 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56367d6-f975-4053-b16f-6050aee1d664 None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:34.054647 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56367d6-f975-4053-b16f-6050aee1d664 None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:34.054962 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56367d6-f975-4053-b16f-6050aee1d664 None admin] PATH_INFO: `/v3/projects/ec187f354d154dc98e045921aa4ca7fd` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:34.056094 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56367d6-f975-4053-b16f-6050aee1d664 None admin] RBAC: Authorizing `identity:get_project(project_id=ec187f354d154dc98e045921aa4ca7fd)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:34.057544 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56367d6-f975-4053-b16f-6050aee1d664 None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:34.059217 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 212/453] 10.222.0.46 () {56 vars in 1186 bytes} [Mon Oct 10 11:46:33 2022] GET /identity/v3/projects/ec187f354d154dc98e045921aa4ca7fd => generated 344 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:46:34.065565 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4090a1-83dc-462d-ba2a-d9d9ba87a836 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:34.162648 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4090a1-83dc-462d-ba2a-d9d9ba87a836 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:34.195698 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0608ce72-4365-4f52-a63f-dd0d9b0046f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:34.196326 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0608ce72-4365-4f52-a63f-dd0d9b0046f1 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:34.196609 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0608ce72-4365-4f52-a63f-dd0d9b0046f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:34.196884 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0608ce72-4365-4f52-a63f-dd0d9b0046f1 None admin] PATH_INFO: `/v3/groups/764cc3a3ef9645fbbe9183049fcbf504` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:34.197984 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0608ce72-4365-4f52-a63f-dd0d9b0046f1 None admin] RBAC: Authorizing `identity:get_group(group_id=764cc3a3ef9645fbbe9183049fcbf504)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:34.199442 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0608ce72-4365-4f52-a63f-dd0d9b0046f1 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:34.201035 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 242/454] 10.222.0.46 () {56 vars in 1180 bytes} [Mon Oct 10 11:46:34 2022] GET /identity/v3/groups/764cc3a3ef9645fbbe9183049fcbf504 => generated 229 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:46:34.207014 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56367d6-f975-4053-b16f-6050aee1d664 None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:34.309178 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56367d6-f975-4053-b16f-6050aee1d664 None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:34.322489 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:34.343588 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbc7f93-652e-461c-90ca-1b5e65261627 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:34.344211 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbc7f93-652e-461c-90ca-1b5e65261627 None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:34.344491 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbc7f93-652e-461c-90ca-1b5e65261627 None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:34.344762 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbc7f93-652e-461c-90ca-1b5e65261627 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:34.345496 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fbc7f93-652e-461c-90ca-1b5e65261627 None admin] RBAC: Adding query filter params (group.id=764cc3a3ef9645fbbe9183049fcbf504, role.id=0d822a0674b6495fac623877a3b38025, scope.project.id=ec187f354d154dc98e045921aa4ca7fd) {{(pid=81676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:46:34.345805 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fbc7f93-652e-461c-90ca-1b5e65261627 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:34.347293 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fbc7f93-652e-461c-90ca-1b5e65261627 None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:34.353693 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 213/455] 10.222.0.46 () {56 vars in 1376 bytes} [Mon Oct 10 11:46:34 2022] GET /identity/v3/role_assignments?group.id=764cc3a3ef9645fbbe9183049fcbf504&scope.project.id=ec187f354d154dc98e045921aa4ca7fd&role.id=0d822a0674b6495fac623877a3b38025 => generated 611 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:46:37.563853 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada77312-eaae-41d5-928e-dcdfb2bb0566 None None] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:37.564671 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada77312-eaae-41d5-928e-dcdfb2bb0566 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:37.564671 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada77312-eaae-41d5-928e-dcdfb2bb0566 None None] PATH_INFO: `/` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:37.566327 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 243/456] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 10 11:46:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:46:37.572123 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502fc294-39df-4904-9815-18b0fd7f32c8 None None] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:37.572889 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502fc294-39df-4904-9815-18b0fd7f32c8 None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:37.572889 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502fc294-39df-4904-9815-18b0fd7f32c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:37.603843 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.auth.core [None req-502fc294-39df-4904-9815-18b0fd7f32c8 None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:46:37.605169 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-502fc294-39df-4904-9815-18b0fd7f32c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:46:37.701935 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 214/457] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 10 11:46:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 10 11:46:38.080778 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af53becc-7db6-43b4-b360-ba8a5d9efff3 None None] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:38.081486 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af53becc-7db6-43b4-b360-ba8a5d9efff3 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:38.081983 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af53becc-7db6-43b4-b360-ba8a5d9efff3 None None] PATH_INFO: `/` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:38.084014 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 244/458] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 10 11:46:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:46:38.090338 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481b273a-cb3f-4cb0-934a-25dd4cec8a07 None None] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:38.091063 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481b273a-cb3f-4cb0-934a-25dd4cec8a07 None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:38.091586 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481b273a-cb3f-4cb0-934a-25dd4cec8a07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:38.119855 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.auth.core [None req-481b273a-cb3f-4cb0-934a-25dd4cec8a07 None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:46:38.121865 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-481b273a-cb3f-4cb0-934a-25dd4cec8a07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:46:38.195738 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af53becc-7db6-43b4-b360-ba8a5d9efff3 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:38.241430 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 215/459] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 10 11:46:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 10 11:46:38.285011 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:38.382470 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92699048-f56c-476f-b189-30610a205f19 None None] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:38.383417 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92699048-f56c-476f-b189-30610a205f19 None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:38.386750 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92699048-f56c-476f-b189-30610a205f19 None None] PATH_INFO: `/` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:38.390911 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 216/460] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 10 11:46:38 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:46:38.394195 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe176a9-d937-41df-b490-f308f2702fa8 None None] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:38.394785 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe176a9-d937-41df-b490-f308f2702fa8 None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:38.395333 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe176a9-d937-41df-b490-f308f2702fa8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:38.408912 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af53becc-7db6-43b4-b360-ba8a5d9efff3 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:38.448482 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.auth.core [None req-3fe176a9-d937-41df-b490-f308f2702fa8 None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:46:38.450602 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-3fe176a9-d937-41df-b490-f308f2702fa8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:46:38.520653 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2f2fe3-fc65-436d-9548-a753982649cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:38.521775 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2f2fe3-fc65-436d-9548-a753982649cf None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:38.522240 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2f2fe3-fc65-436d-9548-a753982649cf None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:38.522697 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2f2fe3-fc65-436d-9548-a753982649cf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:38.542117 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2f2fe3-fc65-436d-9548-a753982649cf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:38.545106 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:38.545891 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2f2fe3-fc65-436d-9548-a753982649cf None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:38.577831 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application [None req-ff2f2fe3-fc65-436d-9548-a753982649cf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:46:38.577831 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:46:38.577831 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:46:38.577831 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:46:38.577831 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:46:38.577831 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:46:38.577831 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:46:38.577831 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:46:38.577831 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:46:38.577831 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:46:38.577831 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:46:38.577831 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:46:38.577831 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 10 11:46:38.577831 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 10 11:46:38.577831 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 10 11:46:38.577831 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 10 11:46:38.580550 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:46:38.580550 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:46:38.580550 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:46:38.580550 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:46:38.580550 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:46:38.580550 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:46:38.580550 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:46:38.580550 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application with Lock( Oct 10 11:46:38.580550 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:46:38.580550 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:46:38.580550 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:46:38.580550 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:46:38.580550 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:46:38.580550 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:46:38.580550 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:46:38.580550 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:46:38.580550 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 10 11:46:38.582749 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 10 11:46:38.582749 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:46:38.582749 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Oct 10 11:46:38.586931 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 245/461] 10.222.0.46 () {56 vars in 1108 bytes} [Mon Oct 10 11:46:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 391 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:46:38.589290 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbe072b-1fdc-4ee6-b812-ff489a1d9f15 None None] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:38.590434 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbe072b-1fdc-4ee6-b812-ff489a1d9f15 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:38.591091 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbe072b-1fdc-4ee6-b812-ff489a1d9f15 None None] PATH_INFO: `/` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:38.594537 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 246/462] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 10 11:46:38 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:46:38.605213 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145adb86-de56-4a57-a1ff-8cb1bb3ab109 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:38.605840 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145adb86-de56-4a57-a1ff-8cb1bb3ab109 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:38.606346 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145adb86-de56-4a57-a1ff-8cb1bb3ab109 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:38.614458 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 217/463] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 10 11:46:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 10 11:46:38.616064 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe176a9-d937-41df-b490-f308f2702fa8 None None] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:38.638653 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-145adb86-de56-4a57-a1ff-8cb1bb3ab109 None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:46:38.641226 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-145adb86-de56-4a57-a1ff-8cb1bb3ab109 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:46:38.776128 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 247/464] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 10 11:46:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 10 11:46:38.777026 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145adb86-de56-4a57-a1ff-8cb1bb3ab109 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:38.791907 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe176a9-d937-41df-b490-f308f2702fa8 None None] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:38.802167 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:38.899092 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a7a4a4-2c5a-49aa-8eb6-1700e6637674 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:38.900458 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a7a4a4-2c5a-49aa-8eb6-1700e6637674 None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:38.901665 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a7a4a4-2c5a-49aa-8eb6-1700e6637674 None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:38.902353 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a7a4a4-2c5a-49aa-8eb6-1700e6637674 None admin] PATH_INFO: `/v3/domains` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:38.904186 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a7a4a4-2c5a-49aa-8eb6-1700e6637674 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:46:38.905913 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a7a4a4-2c5a-49aa-8eb6-1700e6637674 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:38.908699 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145adb86-de56-4a57-a1ff-8cb1bb3ab109 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:38.909079 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a7a4a4-2c5a-49aa-8eb6-1700e6637674 None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:38.939159 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 218/465] 10.222.0.46 () {56 vars in 1109 bytes} [Mon Oct 10 11:46:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 324 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:46:38.947721 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a7a4a4-2c5a-49aa-8eb6-1700e6637674 None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:38.993657 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e214ac75-e028-4dd9-bcff-757361ae0162 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:38.994749 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e214ac75-e028-4dd9-bcff-757361ae0162 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:38.995224 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e214ac75-e028-4dd9-bcff-757361ae0162 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:38.995743 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e214ac75-e028-4dd9-bcff-757361ae0162 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:39.018093 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e214ac75-e028-4dd9-bcff-757361ae0162 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:39.020926 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e214ac75-e028-4dd9-bcff-757361ae0162 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:39.043183 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:39.056899 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application [None req-e214ac75-e028-4dd9-bcff-757361ae0162 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:46:39.056899 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:46:39.056899 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:46:39.056899 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:46:39.056899 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:46:39.056899 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:46:39.056899 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:46:39.056899 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:46:39.056899 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:46:39.056899 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:46:39.056899 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:46:39.056899 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:46:39.056899 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 10 11:46:39.056899 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 10 11:46:39.056899 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 10 11:46:39.056899 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 10 11:46:39.058656 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:46:39.058656 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:46:39.058656 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:46:39.058656 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:46:39.058656 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:46:39.058656 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:46:39.058656 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:46:39.058656 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application with Lock( Oct 10 11:46:39.058656 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:46:39.058656 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:46:39.058656 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:46:39.058656 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:46:39.058656 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:46:39.058656 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:46:39.058656 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:46:39.058656 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:46:39.058656 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 10 11:46:39.060060 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 10 11:46:39.060060 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:46:39.060060 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Oct 10 11:46:39.060060 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 248/466] 10.222.0.46 () {56 vars in 1108 bytes} [Mon Oct 10 11:46:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:46:39.068967 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e214ac75-e028-4dd9-bcff-757361ae0162 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:39.214823 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a7a4a4-2c5a-49aa-8eb6-1700e6637674 None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:39.223733 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:39.261987 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e214ac75-e028-4dd9-bcff-757361ae0162 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:39.285143 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07cbd172-473e-43a0-81b4-0f6cc6e08773 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:39.286178 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cbd172-473e-43a0-81b4-0f6cc6e08773 None admin] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:39.286692 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cbd172-473e-43a0-81b4-0f6cc6e08773 None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:39.287184 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cbd172-473e-43a0-81b4-0f6cc6e08773 None admin] PATH_INFO: `/v3/users` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:39.288373 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07cbd172-473e-43a0-81b4-0f6cc6e08773 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:39.291119 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07cbd172-473e-43a0-81b4-0f6cc6e08773 None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:39.302203 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01cabd42-310f-4efe-a914-eee735b9ca4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:39.303139 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cabd42-310f-4efe-a914-eee735b9ca4d None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:39.303657 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cabd42-310f-4efe-a914-eee735b9ca4d None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:39.304232 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cabd42-310f-4efe-a914-eee735b9ca4d None admin] PATH_INFO: `/v3/domains` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:39.305089 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01cabd42-310f-4efe-a914-eee735b9ca4d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:46:39.305604 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01cabd42-310f-4efe-a914-eee735b9ca4d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:39.307421 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01cabd42-310f-4efe-a914-eee735b9ca4d None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:39.328946 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 249/467] 10.222.0.46 () {56 vars in 1109 bytes} [Mon Oct 10 11:46:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:46:39.330052 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01cabd42-310f-4efe-a914-eee735b9ca4d None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:39.424746 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 219/468] 10.222.0.46 () {60 vars in 1132 bytes} [Mon Oct 10 11:46:38 2022] POST /identity/v3/users => generated 248 bytes in 478 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 11:46:39.426632 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07cbd172-473e-43a0-81b4-0f6cc6e08773 None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:39.467358 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01cabd42-310f-4efe-a914-eee735b9ca4d None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:39.500928 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:39.509997 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ccf676-99bb-487d-8253-312365f86324 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:39.510685 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ccf676-99bb-487d-8253-312365f86324 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:39.510990 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ccf676-99bb-487d-8253-312365f86324 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:39.511265 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ccf676-99bb-487d-8253-312365f86324 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:39.524925 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ccf676-99bb-487d-8253-312365f86324 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:39.526534 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ccf676-99bb-487d-8253-312365f86324 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:39.553258 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application [None req-a0ccf676-99bb-487d-8253-312365f86324 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:46:39.553258 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:46:39.553258 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:46:39.553258 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:46:39.553258 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:46:39.553258 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:46:39.553258 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:46:39.553258 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:46:39.553258 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:46:39.553258 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:46:39.553258 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:46:39.553258 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:46:39.553258 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 10 11:46:39.553258 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 10 11:46:39.553258 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 10 11:46:39.553258 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 10 11:46:39.554979 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:46:39.554979 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:46:39.554979 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:46:39.554979 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:46:39.554979 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:46:39.554979 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:46:39.554979 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:46:39.554979 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application with Lock( Oct 10 11:46:39.554979 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:46:39.554979 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:46:39.554979 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:46:39.554979 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:46:39.554979 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:46:39.554979 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:46:39.554979 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:46:39.554979 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:46:39.554979 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 10 11:46:39.556599 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 10 11:46:39.556599 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:46:39.556599 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Oct 10 11:46:39.566705 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 250/469] 10.222.0.46 () {56 vars in 1108 bytes} [Mon Oct 10 11:46:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:46:39.569158 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ccf676-99bb-487d-8253-312365f86324 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:39.588344 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:39.609779 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07cbd172-473e-43a0-81b4-0f6cc6e08773 None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:39.653154 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4866ec-8fd6-4c43-91ec-84219d50bc95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:39.654134 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4866ec-8fd6-4c43-91ec-84219d50bc95 None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:39.654134 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4866ec-8fd6-4c43-91ec-84219d50bc95 None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:39.654403 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4866ec-8fd6-4c43-91ec-84219d50bc95 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:39.672419 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4866ec-8fd6-4c43-91ec-84219d50bc95 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:39.673928 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4866ec-8fd6-4c43-91ec-84219d50bc95 None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:39.695235 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application [None req-3a4866ec-8fd6-4c43-91ec-84219d50bc95 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:46:39.695235 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:46:39.695235 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:46:39.695235 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:46:39.695235 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:46:39.695235 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:46:39.695235 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:46:39.695235 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:46:39.695235 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:46:39.695235 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:46:39.695235 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:46:39.695235 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:46:39.695235 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 10 11:46:39.695235 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 10 11:46:39.695235 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 10 11:46:39.695235 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 10 11:46:39.696910 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:46:39.696910 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:46:39.696910 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:46:39.696910 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:46:39.696910 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:46:39.696910 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:46:39.696910 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:46:39.696910 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application with Lock( Oct 10 11:46:39.696910 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:46:39.696910 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:46:39.696910 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:46:39.696910 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:46:39.696910 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:46:39.696910 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:46:39.696910 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:46:39.696910 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:46:39.696910 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 10 11:46:39.698467 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 10 11:46:39.698467 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:46:39.698467 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Oct 10 11:46:39.698467 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 220/470] 10.222.0.46 () {56 vars in 1108 bytes} [Mon Oct 10 11:46:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:46:39.698467 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4866ec-8fd6-4c43-91ec-84219d50bc95 None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:39.776577 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ccf676-99bb-487d-8253-312365f86324 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:39.843945 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4866ec-8fd6-4c43-91ec-84219d50bc95 None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:39.871771 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611621fc-7b9d-4d27-baa2-8bce566ba7ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:39.872404 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611621fc-7b9d-4d27-baa2-8bce566ba7ea None admin] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:39.872687 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611621fc-7b9d-4d27-baa2-8bce566ba7ea None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:39.872976 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611621fc-7b9d-4d27-baa2-8bce566ba7ea None admin] PATH_INFO: `/v3/users` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:39.873630 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611621fc-7b9d-4d27-baa2-8bce566ba7ea None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:39.875756 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611621fc-7b9d-4d27-baa2-8bce566ba7ea None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:39.902174 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79c3069-3171-4f5a-87fd-299fa26ddf0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:39.903265 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79c3069-3171-4f5a-87fd-299fa26ddf0a None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:39.903768 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79c3069-3171-4f5a-87fd-299fa26ddf0a None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:39.904262 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79c3069-3171-4f5a-87fd-299fa26ddf0a None admin] PATH_INFO: `/v3/domains` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:39.905430 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79c3069-3171-4f5a-87fd-299fa26ddf0a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:46:39.905992 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79c3069-3171-4f5a-87fd-299fa26ddf0a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:39.908404 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79c3069-3171-4f5a-87fd-299fa26ddf0a None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:39.911300 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:39.935922 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 221/471] 10.222.0.46 () {56 vars in 1109 bytes} [Mon Oct 10 11:46:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:46:39.935922 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79c3069-3171-4f5a-87fd-299fa26ddf0a None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:39.956610 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 251/472] 10.222.0.46 () {60 vars in 1132 bytes} [Mon Oct 10 11:46:39 2022] POST /identity/v3/users => generated 249 bytes in 388 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 11:46:39.965007 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611621fc-7b9d-4d27-baa2-8bce566ba7ea None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:40.049246 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:40.128249 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79c3069-3171-4f5a-87fd-299fa26ddf0a None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:40.142851 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611621fc-7b9d-4d27-baa2-8bce566ba7ea None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:40.178498 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9c2716-348d-46c6-bbac-5f26be2db104 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:40.179153 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9c2716-348d-46c6-bbac-5f26be2db104 None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:40.180441 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9c2716-348d-46c6-bbac-5f26be2db104 None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:40.180441 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9c2716-348d-46c6-bbac-5f26be2db104 None admin] PATH_INFO: `/v3/domains` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:40.180441 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9c2716-348d-46c6-bbac-5f26be2db104 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:46:40.180682 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9c2716-348d-46c6-bbac-5f26be2db104 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:40.182716 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9c2716-348d-46c6-bbac-5f26be2db104 None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:40.194921 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 222/473] 10.222.0.46 () {56 vars in 1109 bytes} [Mon Oct 10 11:46:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:46:40.196344 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8831fed9-51a2-4b91-bb66-3835c3d9e58d None None] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:40.196630 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8831fed9-51a2-4b91-bb66-3835c3d9e58d None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:40.196902 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8831fed9-51a2-4b91-bb66-3835c3d9e58d None None] PATH_INFO: `/` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:40.202513 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 223/474] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 10 11:46:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:46:40.202513 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8831fed9-51a2-4b91-bb66-3835c3d9e58d None None] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:40.203961 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a570688f-a18e-4a8b-a8c0-c9f3c296fe6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:40.204683 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a570688f-a18e-4a8b-a8c0-c9f3c296fe6b None admin] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:40.205081 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a570688f-a18e-4a8b-a8c0-c9f3c296fe6b None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:40.206327 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a570688f-a18e-4a8b-a8c0-c9f3c296fe6b None admin] PATH_INFO: `/v3/users` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:40.207051 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a570688f-a18e-4a8b-a8c0-c9f3c296fe6b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:40.208529 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a570688f-a18e-4a8b-a8c0-c9f3c296fe6b None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:40.274656 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:40.332139 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8831fed9-51a2-4b91-bb66-3835c3d9e58d None None] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:40.338869 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 252/475] 10.222.0.46 () {60 vars in 1132 bytes} [Mon Oct 10 11:46:39 2022] POST /identity/v3/users => generated 248 bytes in 375 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 11:46:40.340438 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e416b2a-c7cc-46fc-b52b-5e5893084b16 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:40.340729 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e416b2a-c7cc-46fc-b52b-5e5893084b16 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:40.341070 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e416b2a-c7cc-46fc-b52b-5e5893084b16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:40.382437 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4790594d-de8b-4b4e-b244-9028421337a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:40.383205 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4790594d-de8b-4b4e-b244-9028421337a6 None admin] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:40.383513 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4790594d-de8b-4b4e-b244-9028421337a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:40.383775 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4790594d-de8b-4b4e-b244-9028421337a6 None admin] PATH_INFO: `/v3/users` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:40.384054 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-5e416b2a-c7cc-46fc-b52b-5e5893084b16 None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:46:40.384460 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4790594d-de8b-4b4e-b244-9028421337a6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:40.385327 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-5e416b2a-c7cc-46fc-b52b-5e5893084b16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:46:40.386014 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4790594d-de8b-4b4e-b244-9028421337a6 None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:40.488978 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 253/476] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 10 11:46:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 10 11:46:40.501640 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 224/477] 10.222.0.46 () {60 vars in 1132 bytes} [Mon Oct 10 11:46:40 2022] POST /identity/v3/users => generated 246 bytes in 301 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 10 11:46:40.800440 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e416b2a-c7cc-46fc-b52b-5e5893084b16 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:40.831156 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:40.933852 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e416b2a-c7cc-46fc-b52b-5e5893084b16 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:40.976094 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1076362a-cca0-42d5-a9bb-a12e3a3bb050 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:40.976726 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1076362a-cca0-42d5-a9bb-a12e3a3bb050 None admin] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:40.977008 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1076362a-cca0-42d5-a9bb-a12e3a3bb050 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:40.977290 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1076362a-cca0-42d5-a9bb-a12e3a3bb050 None admin] PATH_INFO: `/v3/roles` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:40.978032 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1076362a-cca0-42d5-a9bb-a12e3a3bb050 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:40.979511 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1076362a-cca0-42d5-a9bb-a12e3a3bb050 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:41.027607 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.sql.core [None req-1076362a-cca0-42d5-a9bb-a12e3a3bb050 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Oct 10 11:46:41.027607 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Oct 10 11:46:41.027607 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [parameters: {'id': '75782df6d56c4324b5a963f8ccf3c2c1', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Oct 10 11:46:41.027607 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81677) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Oct 10 11:46:41.049075 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application [None req-1076362a-cca0-42d5-a9bb-a12e3a3bb050 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Oct 10 11:46:41.049075 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:46:41.049075 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 10 11:46:41.049075 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 10 11:46:41.049075 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 10 11:46:41.049075 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 10 11:46:41.049075 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 10 11:46:41.049075 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application result = self._query(query) Oct 10 11:46:41.049075 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 10 11:46:41.049075 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application conn.query(q) Oct 10 11:46:41.049075 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 10 11:46:41.049075 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 10 11:46:41.049075 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 10 11:46:41.049075 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application result.read() Oct 10 11:46:41.049075 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 10 11:46:41.050812 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 10 11:46:41.050812 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 10 11:46:41.050812 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 10 11:46:41.050812 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 10 11:46:41.050812 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 10 11:46:41.050812 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 10 11:46:41.050812 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 10 11:46:41.050812 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Oct 10 11:46:41.050812 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Oct 10 11:46:41.050812 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Oct 10 11:46:41.050812 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Oct 10 11:46:41.050812 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:46:41.050812 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Oct 10 11:46:41.050812 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return method(*args, **kwargs) Oct 10 11:46:41.050812 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Oct 10 11:46:41.050812 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return ref.to_dict() Oct 10 11:46:41.050812 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 10 11:46:41.050812 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application next(self.gen) Oct 10 11:46:41.052615 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Oct 10 11:46:41.052615 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application yield resource Oct 10 11:46:41.052615 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 10 11:46:41.052615 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application next(self.gen) Oct 10 11:46:41.052615 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Oct 10 11:46:41.052615 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application self.session.rollback() Oct 10 11:46:41.052615 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 10 11:46:41.052615 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application self.force_reraise() Oct 10 11:46:41.052615 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 10 11:46:41.052615 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise self.value Oct 10 11:46:41.052615 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Oct 10 11:46:41.052615 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Oct 10 11:46:41.052615 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Oct 10 11:46:41.052615 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application session.commit() Oct 10 11:46:41.052615 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Oct 10 11:46:41.052615 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Oct 10 11:46:41.052615 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Oct 10 11:46:41.054432 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application self._prepare_impl() Oct 10 11:46:41.054432 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Oct 10 11:46:41.054432 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application self.session.flush() Oct 10 11:46:41.054432 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Oct 10 11:46:41.054432 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application self._flush(objects) Oct 10 11:46:41.054432 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Oct 10 11:46:41.054432 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Oct 10 11:46:41.054432 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Oct 10 11:46:41.054432 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application compat.raise_( Oct 10 11:46:41.054432 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 10 11:46:41.054432 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception Oct 10 11:46:41.054432 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Oct 10 11:46:41.054432 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application flush_context.execute() Oct 10 11:46:41.054432 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Oct 10 11:46:41.054432 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application rec.execute(self) Oct 10 11:46:41.054432 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Oct 10 11:46:41.054432 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Oct 10 11:46:41.058600 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Oct 10 11:46:41.058600 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application _emit_insert_statements( Oct 10 11:46:41.058600 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Oct 10 11:46:41.058600 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application c = connection._execute_20( Oct 10 11:46:41.058600 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Oct 10 11:46:41.058600 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Oct 10 11:46:41.058600 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Oct 10 11:46:41.058600 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Oct 10 11:46:41.058600 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Oct 10 11:46:41.058600 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application ret = self._execute_context( Oct 10 11:46:41.058600 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Oct 10 11:46:41.058600 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Oct 10 11:46:41.058600 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Oct 10 11:46:41.058600 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Oct 10 11:46:41.058600 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 10 11:46:41.058600 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception Oct 10 11:46:41.062216 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 10 11:46:41.062216 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 10 11:46:41.062216 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 10 11:46:41.062216 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 10 11:46:41.062216 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 10 11:46:41.062216 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application result = self._query(query) Oct 10 11:46:41.062216 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 10 11:46:41.062216 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application conn.query(q) Oct 10 11:46:41.062216 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 10 11:46:41.062216 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 10 11:46:41.062216 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 10 11:46:41.062216 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application result.read() Oct 10 11:46:41.062216 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 10 11:46:41.062216 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 10 11:46:41.062216 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 10 11:46:41.062216 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 10 11:46:41.073202 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 10 11:46:41.073202 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 10 11:46:41.073202 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 10 11:46:41.073202 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 10 11:46:41.073202 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Oct 10 11:46:41.073202 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Oct 10 11:46:41.073202 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application [parameters: {'id': '75782df6d56c4324b5a963f8ccf3c2c1', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Oct 10 11:46:41.073202 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Oct 10 11:46:41.073202 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Oct 10 11:46:41.073202 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 10 11:46:41.073202 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Oct 10 11:46:41.073202 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:46:41.073202 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:46:41.073202 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:46:41.073202 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:46:41.073202 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:46:41.076416 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:46:41.076416 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:46:41.076416 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:46:41.076416 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:46:41.076416 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:46:41.076416 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:46:41.076416 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Oct 10 11:46:41.076416 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Oct 10 11:46:41.076416 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:46:41.076416 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:46:41.076416 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Oct 10 11:46:41.076416 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Oct 10 11:46:41.076416 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Oct 10 11:46:41.076416 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Oct 10 11:46:41.076416 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Oct 10 11:46:41.076416 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Oct 10 11:46:41.084280 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 254/478] 10.222.0.46 () {60 vars in 1131 bytes} [Mon Oct 10 11:46:40 2022] POST /identity/v3/roles => generated 144 bytes in 251 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:46:41.084505 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4790594d-de8b-4b4e-b244-9028421337a6 None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:41.186068 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4790594d-de8b-4b4e-b244-9028421337a6 None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:41.198545 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:41.227357 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4990853a-65f5-40b8-b0a4-6c4ad3700937 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:41.228015 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4990853a-65f5-40b8-b0a4-6c4ad3700937 None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:41.228301 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4990853a-65f5-40b8-b0a4-6c4ad3700937 None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:41.228563 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4990853a-65f5-40b8-b0a4-6c4ad3700937 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:41.242962 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4990853a-65f5-40b8-b0a4-6c4ad3700937 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 10 11:46:41.248328 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application [None req-4990853a-65f5-40b8-b0a4-6c4ad3700937 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 10 11:46:41.248328 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:46:41.248328 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:46:41.248328 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:46:41.248328 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:46:41.248328 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:46:41.248328 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:46:41.248328 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:46:41.248328 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:46:41.248328 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:46:41.248328 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:46:41.248328 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:46:41.248328 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 10 11:46:41.248328 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 10 11:46:41.248328 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 10 11:46:41.248328 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 10 11:46:41.262938 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 10 11:46:41.262938 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 10 11:46:41.262938 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 10 11:46:41.262938 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 10 11:46:41.262938 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:46:41.262938 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:46:41.262938 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:46:41.262938 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:46:41.262938 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:46:41.262938 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:46:41.262938 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:46:41.262938 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application with Lock( Oct 10 11:46:41.262938 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:46:41.262938 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:46:41.262938 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:46:41.262938 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:46:41.265690 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:46:41.265690 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:46:41.265690 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:46:41.265690 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:46:41.265690 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 10 11:46:41.265690 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 10 11:46:41.265690 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 10 11:46:41.265690 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 10 11:46:41.265690 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 10 11:46:41.265690 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 10 11:46:41.265690 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 10 11:46:41.265690 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Oct 10 11:46:41.265690 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 225/479] 10.222.0.46 () {56 vars in 1114 bytes} [Mon Oct 10 11:46:41 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:46:41.267948 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1076362a-cca0-42d5-a9bb-a12e3a3bb050 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:41.358188 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:41.377055 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1076362a-cca0-42d5-a9bb-a12e3a3bb050 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:41.417005 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e510a8cd-7d14-4185-810f-411e2bd08b7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:41.417636 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e510a8cd-7d14-4185-810f-411e2bd08b7a None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:41.417921 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e510a8cd-7d14-4185-810f-411e2bd08b7a None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:41.418206 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e510a8cd-7d14-4185-810f-411e2bd08b7a None admin] PATH_INFO: `/v3/roles` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:41.418958 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e510a8cd-7d14-4185-810f-411e2bd08b7a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:46:41.419267 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e510a8cd-7d14-4185-810f-411e2bd08b7a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:41.420711 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e510a8cd-7d14-4185-810f-411e2bd08b7a None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:41.432386 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 255/480] 10.222.0.46 () {56 vars in 1111 bytes} [Mon Oct 10 11:46:41 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:46:42.103662 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05aa2dcc-882e-4905-9cd1-29d6fa66dcb7 None None] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:42.105176 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05aa2dcc-882e-4905-9cd1-29d6fa66dcb7 None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:42.105959 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05aa2dcc-882e-4905-9cd1-29d6fa66dcb7 None None] PATH_INFO: `/` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:42.109338 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 226/481] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 10 11:46:42 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:46:42.117894 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c69d4c6-adb1-4692-819b-8e4083b5c1b7 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:42.119173 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c69d4c6-adb1-4692-819b-8e4083b5c1b7 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:42.119465 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c69d4c6-adb1-4692-819b-8e4083b5c1b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:42.151870 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-2c69d4c6-adb1-4692-819b-8e4083b5c1b7 None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:46:42.153108 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-2c69d4c6-adb1-4692-819b-8e4083b5c1b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:46:42.257364 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 256/482] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 10 11:46:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 10 11:46:42.827932 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886c3896-6604-4263-aaa8-5fe4144fa427 None None] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:42.828158 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886c3896-6604-4263-aaa8-5fe4144fa427 None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:42.828547 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886c3896-6604-4263-aaa8-5fe4144fa427 None None] PATH_INFO: `/` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:42.830166 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 227/483] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 10 11:46:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:46:42.840053 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b3055f-df6b-4591-b617-786e61ab60d7 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:42.840417 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b3055f-df6b-4591-b617-786e61ab60d7 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:42.840634 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b3055f-df6b-4591-b617-786e61ab60d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:42.864564 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-96b3055f-df6b-4591-b617-786e61ab60d7 None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:46:42.865780 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-96b3055f-df6b-4591-b617-786e61ab60d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:46:42.960957 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 257/484] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 10 11:46:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 10 11:46:43.068836 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886c3896-6604-4263-aaa8-5fe4144fa427 None None] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:43.222890 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886c3896-6604-4263-aaa8-5fe4144fa427 None None] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:43.277803 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb23ff3-a328-42b5-b9e2-e4beac8e95ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:43.278470 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb23ff3-a328-42b5-b9e2-e4beac8e95ce None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:43.278784 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb23ff3-a328-42b5-b9e2-e4beac8e95ce None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:43.279176 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb23ff3-a328-42b5-b9e2-e4beac8e95ce None admin] PATH_INFO: `/v3/roles/service` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:43.299881 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb23ff3-a328-42b5-b9e2-e4beac8e95ce None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 10 11:46:43.305073 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application [None req-dcb23ff3-a328-42b5-b9e2-e4beac8e95ce None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 10 11:46:43.305073 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:46:43.305073 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:46:43.305073 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:46:43.305073 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:46:43.305073 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:46:43.305073 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:46:43.305073 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:46:43.305073 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:46:43.305073 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:46:43.305073 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:46:43.305073 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:46:43.305073 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 10 11:46:43.305073 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 10 11:46:43.305073 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 10 11:46:43.305073 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 10 11:46:43.308358 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 10 11:46:43.308358 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 10 11:46:43.308358 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 10 11:46:43.308358 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 10 11:46:43.308358 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:46:43.308358 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:46:43.308358 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:46:43.308358 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:46:43.308358 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:46:43.308358 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:46:43.308358 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:46:43.308358 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application with Lock( Oct 10 11:46:43.308358 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:46:43.308358 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:46:43.308358 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:46:43.308358 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:46:43.311159 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:46:43.311159 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:46:43.311159 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:46:43.311159 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:46:43.311159 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 10 11:46:43.311159 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 10 11:46:43.311159 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 10 11:46:43.311159 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 10 11:46:43.311159 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 10 11:46:43.311159 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 10 11:46:43.311159 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 10 11:46:43.311159 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Oct 10 11:46:43.311159 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 228/485] 10.222.0.46 () {56 vars in 1102 bytes} [Mon Oct 10 11:46:43 2022] GET /identity/v3/roles/service => generated 85 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:46:43.318039 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b3055f-df6b-4591-b617-786e61ab60d7 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:43.347282 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:43.578745 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b3055f-df6b-4591-b617-786e61ab60d7 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:43.658720 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb23ff3-a328-42b5-b9e2-e4beac8e95ce None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:43.660585 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f284afb0-25b1-44a1-8ec2-d7253568d065 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:43.661656 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f284afb0-25b1-44a1-8ec2-d7253568d065 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:43.662100 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f284afb0-25b1-44a1-8ec2-d7253568d065 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:43.662546 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f284afb0-25b1-44a1-8ec2-d7253568d065 None admin] PATH_INFO: `/v3/roles` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:43.663741 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f284afb0-25b1-44a1-8ec2-d7253568d065 None admin] RBAC: Adding query filter params (name=service) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:46:43.664258 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f284afb0-25b1-44a1-8ec2-d7253568d065 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:43.666739 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f284afb0-25b1-44a1-8ec2-d7253568d065 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:43.687666 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 258/486] 10.222.0.46 () {56 vars in 1103 bytes} [Mon Oct 10 11:46:43 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 372 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 11:46:43.698429 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f284afb0-25b1-44a1-8ec2-d7253568d065 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:43.731046 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:43.863552 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:43.917975 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb23ff3-a328-42b5-b9e2-e4beac8e95ce None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:43.931357 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f284afb0-25b1-44a1-8ec2-d7253568d065 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:43.972292 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67528897-f1bd-480e-98ce-d2977ee6e437 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:43.975172 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67528897-f1bd-480e-98ce-d2977ee6e437 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:43.976915 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67528897-f1bd-480e-98ce-d2977ee6e437 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:43.977400 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67528897-f1bd-480e-98ce-d2977ee6e437 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:44.001926 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67528897-f1bd-480e-98ce-d2977ee6e437 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:44.004514 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67528897-f1bd-480e-98ce-d2977ee6e437 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:44.020293 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65f177b-bb56-4ce6-a156-9ca2279afc5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:44.021291 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65f177b-bb56-4ce6-a156-9ca2279afc5d None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:44.021812 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65f177b-bb56-4ce6-a156-9ca2279afc5d None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:44.022284 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65f177b-bb56-4ce6-a156-9ca2279afc5d None admin] PATH_INFO: `/v3/roles/service` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:44.036074 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application [None req-67528897-f1bd-480e-98ce-d2977ee6e437 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:46:44.036074 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:46:44.036074 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:46:44.036074 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:46:44.036074 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:46:44.036074 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:46:44.036074 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:46:44.036074 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:46:44.036074 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:46:44.036074 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:46:44.036074 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:46:44.036074 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:46:44.036074 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 10 11:46:44.036074 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 10 11:46:44.036074 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 10 11:46:44.036074 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 10 11:46:44.037002 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:46:44.037002 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:46:44.037002 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:46:44.037002 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:46:44.037002 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:46:44.037002 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:46:44.037002 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:46:44.037002 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application with Lock( Oct 10 11:46:44.037002 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:46:44.037002 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:46:44.037002 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:46:44.037002 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:46:44.037002 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:46:44.037002 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:46:44.037002 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:46:44.037002 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:46:44.037002 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 10 11:46:44.037821 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 10 11:46:44.037821 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:46:44.037821 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Oct 10 11:46:44.046199 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 259/487] 10.222.0.46 () {56 vars in 1108 bytes} [Mon Oct 10 11:46:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 350 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 10 11:46:44.047073 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65f177b-bb56-4ce6-a156-9ca2279afc5d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 10 11:46:44.052315 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application [None req-a65f177b-bb56-4ce6-a156-9ca2279afc5d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 10 11:46:44.052315 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:46:44.052315 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:46:44.052315 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:46:44.052315 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:46:44.052315 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:46:44.052315 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:46:44.052315 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:46:44.052315 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:46:44.052315 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:46:44.052315 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:46:44.052315 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:46:44.052315 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 10 11:46:44.052315 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 10 11:46:44.052315 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 10 11:46:44.052315 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 10 11:46:44.053196 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 10 11:46:44.053196 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 10 11:46:44.053196 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 10 11:46:44.053196 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 10 11:46:44.053196 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:46:44.053196 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:46:44.053196 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:46:44.053196 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:46:44.053196 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:46:44.053196 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:46:44.053196 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:46:44.053196 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application with Lock( Oct 10 11:46:44.053196 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:46:44.053196 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:46:44.053196 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:46:44.053196 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:46:44.053996 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:46:44.053996 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:46:44.053996 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:46:44.053996 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:46:44.053996 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 10 11:46:44.053996 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 10 11:46:44.053996 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 10 11:46:44.053996 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 10 11:46:44.053996 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 10 11:46:44.053996 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 10 11:46:44.053996 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 10 11:46:44.053996 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Oct 10 11:46:44.056119 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67528897-f1bd-480e-98ce-d2977ee6e437 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:44.065198 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 229/488] 10.222.0.46 () {56 vars in 1102 bytes} [Mon Oct 10 11:46:43 2022] GET /identity/v3/roles/service => generated 85 bytes in 407 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:46:44.073368 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65f177b-bb56-4ce6-a156-9ca2279afc5d None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:44.195566 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67528897-f1bd-480e-98ce-d2977ee6e437 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:44.226484 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:44.246369 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5380fe5d-0c5c-4477-a554-1cba0ef86167 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:44.247671 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5380fe5d-0c5c-4477-a554-1cba0ef86167 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:44.249484 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5380fe5d-0c5c-4477-a554-1cba0ef86167 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:44.250224 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5380fe5d-0c5c-4477-a554-1cba0ef86167 None admin] PATH_INFO: `/v3/domains` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:44.252088 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5380fe5d-0c5c-4477-a554-1cba0ef86167 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:46:44.252867 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5380fe5d-0c5c-4477-a554-1cba0ef86167 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:44.255906 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5380fe5d-0c5c-4477-a554-1cba0ef86167 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:44.286457 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 260/489] 10.222.0.46 () {56 vars in 1109 bytes} [Mon Oct 10 11:46:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:46:44.294406 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5380fe5d-0c5c-4477-a554-1cba0ef86167 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:44.329811 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:44.347911 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65f177b-bb56-4ce6-a156-9ca2279afc5d None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:44.421270 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cdf9aff-b581-45e9-82d2-49b3a2296c9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:44.423002 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdf9aff-b581-45e9-82d2-49b3a2296c9e None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:44.423508 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdf9aff-b581-45e9-82d2-49b3a2296c9e None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:44.424374 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdf9aff-b581-45e9-82d2-49b3a2296c9e None admin] PATH_INFO: `/v3/roles` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:44.425685 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cdf9aff-b581-45e9-82d2-49b3a2296c9e None admin] RBAC: Adding query filter params (name=service) {{(pid=81676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:46:44.426595 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cdf9aff-b581-45e9-82d2-49b3a2296c9e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:44.429935 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cdf9aff-b581-45e9-82d2-49b3a2296c9e None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:44.449332 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5380fe5d-0c5c-4477-a554-1cba0ef86167 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:44.454712 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 230/490] 10.222.0.46 () {56 vars in 1103 bytes} [Mon Oct 10 11:46:44 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 379 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 11:46:44.454712 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a7e412-7c07-42f5-a14a-82623529968e None None] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:44.454712 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a7e412-7c07-42f5-a14a-82623529968e None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:44.454712 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a7e412-7c07-42f5-a14a-82623529968e None None] PATH_INFO: `/` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:44.459127 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 231/491] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 10 11:46:44 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:46:44.465806 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a7e412-7c07-42f5-a14a-82623529968e None None] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:44.515236 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04244ef9-430c-4e9a-a7ca-5be389119a56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:44.516582 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04244ef9-430c-4e9a-a7ca-5be389119a56 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:44.517077 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04244ef9-430c-4e9a-a7ca-5be389119a56 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:44.518143 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04244ef9-430c-4e9a-a7ca-5be389119a56 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:44.527752 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04244ef9-430c-4e9a-a7ca-5be389119a56 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:44.530413 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04244ef9-430c-4e9a-a7ca-5be389119a56 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:44.546231 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application [None req-04244ef9-430c-4e9a-a7ca-5be389119a56 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Oct 10 11:46:44.546231 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:46:44.546231 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:46:44.546231 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:46:44.546231 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:46:44.546231 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:46:44.546231 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:46:44.546231 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:46:44.546231 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:46:44.546231 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:46:44.546231 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:46:44.546231 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:46:44.546231 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 10 11:46:44.546231 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 10 11:46:44.546231 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 10 11:46:44.546231 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 10 11:46:44.547410 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:46:44.547410 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:46:44.547410 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 10 11:46:44.547410 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 10 11:46:44.547410 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 10 11:46:44.547410 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 10 11:46:44.547410 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:46:44.547410 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:46:44.547410 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:46:44.547410 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:46:44.547410 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:46:44.547410 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application with Lock( Oct 10 11:46:44.547410 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:46:44.547410 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:46:44.547410 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:46:44.547410 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:46:44.547410 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:46:44.547410 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:46:44.548281 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:46:44.548281 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:46:44.548281 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 10 11:46:44.548281 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 10 11:46:44.548281 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 10 11:46:44.548281 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 10 11:46:44.548281 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 10 11:46:44.548281 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 10 11:46:44.548281 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Oct 10 11:46:44.548281 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Oct 10 11:46:44.549210 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 261/492] 10.222.0.46 () {56 vars in 1099 bytes} [Mon Oct 10 11:46:44 2022] GET /identity/v3/users/cinder => generated 84 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 10 11:46:44.553555 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348f6ea1-b14a-45b5-b65b-bddb69c37a7e None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:44.553555 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348f6ea1-b14a-45b5-b65b-bddb69c37a7e None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:44.553555 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348f6ea1-b14a-45b5-b65b-bddb69c37a7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:44.580741 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-348f6ea1-b14a-45b5-b65b-bddb69c37a7e None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:46:44.582801 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-348f6ea1-b14a-45b5-b65b-bddb69c37a7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:46:44.646793 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:44.659219 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a7e412-7c07-42f5-a14a-82623529968e None None] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:44.703119 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 262/493] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 10 11:46:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 10 11:46:44.705725 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521c1077-5d63-489b-abb1-25e481ecd856 None None] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:44.706357 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521c1077-5d63-489b-abb1-25e481ecd856 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:44.706919 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521c1077-5d63-489b-abb1-25e481ecd856 None None] PATH_INFO: `/` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:44.709394 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 263/494] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 10 11:46:44 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:46:44.710429 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521c1077-5d63-489b-abb1-25e481ecd856 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:44.729794 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5dc64f8-c1f9-4fc6-8c2d-b555d0dfe2eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:44.730920 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5dc64f8-c1f9-4fc6-8c2d-b555d0dfe2eb None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:44.731417 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5dc64f8-c1f9-4fc6-8c2d-b555d0dfe2eb None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:44.731891 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5dc64f8-c1f9-4fc6-8c2d-b555d0dfe2eb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:44.751427 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5dc64f8-c1f9-4fc6-8c2d-b555d0dfe2eb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:44.754549 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5dc64f8-c1f9-4fc6-8c2d-b555d0dfe2eb None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:44.782962 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application [None req-d5dc64f8-c1f9-4fc6-8c2d-b555d0dfe2eb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:46:44.782962 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:46:44.782962 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:46:44.782962 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:46:44.782962 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:46:44.782962 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:46:44.782962 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:46:44.782962 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:46:44.782962 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:46:44.782962 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:46:44.782962 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:46:44.782962 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:46:44.782962 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 10 11:46:44.782962 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 10 11:46:44.782962 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 10 11:46:44.782962 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 10 11:46:44.783916 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:46:44.783916 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:46:44.783916 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:46:44.783916 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:46:44.783916 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:46:44.783916 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:46:44.783916 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:46:44.783916 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application with Lock( Oct 10 11:46:44.783916 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:46:44.783916 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:46:44.783916 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:46:44.783916 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:46:44.783916 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:46:44.783916 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:46:44.783916 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:46:44.783916 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:46:44.783916 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 10 11:46:44.784733 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 10 11:46:44.784733 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:46:44.784733 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Oct 10 11:46:44.784733 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 232/495] 10.222.0.46 () {56 vars in 1108 bytes} [Mon Oct 10 11:46:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 325 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:46:44.787204 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489c03e8-9e0a-4e3c-b1df-76747609f78e None None] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:44.787376 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489c03e8-9e0a-4e3c-b1df-76747609f78e None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:44.787770 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489c03e8-9e0a-4e3c-b1df-76747609f78e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:44.801287 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:44.820879 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.auth.core [None req-489c03e8-9e0a-4e3c-b1df-76747609f78e None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:46:44.822561 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-489c03e8-9e0a-4e3c-b1df-76747609f78e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:46:44.856837 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521c1077-5d63-489b-abb1-25e481ecd856 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:44.909100 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e664fbf6-2a0d-4316-98c2-6ecf891891a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:44.910684 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e664fbf6-2a0d-4316-98c2-6ecf891891a2 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:44.911196 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e664fbf6-2a0d-4316-98c2-6ecf891891a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:44.911835 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e664fbf6-2a0d-4316-98c2-6ecf891891a2 None admin] PATH_INFO: `/v3/users` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:44.912880 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e664fbf6-2a0d-4316-98c2-6ecf891891a2 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:46:44.913396 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e664fbf6-2a0d-4316-98c2-6ecf891891a2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:44.915411 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e664fbf6-2a0d-4316-98c2-6ecf891891a2 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:44.940343 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 233/496] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 10 11:46:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 10 11:46:44.941561 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489c03e8-9e0a-4e3c-b1df-76747609f78e None None] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:45.017003 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 264/497] 10.222.0.46 () {56 vars in 1137 bytes} [Mon Oct 10 11:46:44 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:46:45.025439 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e664fbf6-2a0d-4316-98c2-6ecf891891a2 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:45.082402 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489c03e8-9e0a-4e3c-b1df-76747609f78e None None] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:45.127116 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03cbf4e-9984-4ec6-841c-177f5e3676e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:45.128248 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03cbf4e-9984-4ec6-841c-177f5e3676e7 None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:45.128997 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:45.129588 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03cbf4e-9984-4ec6-841c-177f5e3676e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:45.130080 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03cbf4e-9984-4ec6-841c-177f5e3676e7 None admin] PATH_INFO: `/v3/domains` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:45.130981 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03cbf4e-9984-4ec6-841c-177f5e3676e7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:46:45.131481 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03cbf4e-9984-4ec6-841c-177f5e3676e7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:45.133080 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03cbf4e-9984-4ec6-841c-177f5e3676e7 None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:45.135001 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:45.147844 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 234/498] 10.222.0.46 () {56 vars in 1109 bytes} [Mon Oct 10 11:46:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:46:45.153980 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03cbf4e-9984-4ec6-841c-177f5e3676e7 None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:45.258275 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e664fbf6-2a0d-4316-98c2-6ecf891891a2 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:45.275231 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03cbf4e-9984-4ec6-841c-177f5e3676e7 None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:45.301588 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a3301f-32e7-4dcf-bbf4-f63930154ce5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:45.302304 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a3301f-32e7-4dcf-bbf4-f63930154ce5 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:45.302582 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a3301f-32e7-4dcf-bbf4-f63930154ce5 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:45.302884 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a3301f-32e7-4dcf-bbf4-f63930154ce5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:45.316939 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb7fc8d-5ac2-4027-bf28-8c33c64fc4a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:45.318010 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb7fc8d-5ac2-4027-bf28-8c33c64fc4a0 None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:45.318504 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb7fc8d-5ac2-4027-bf28-8c33c64fc4a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:45.318979 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb7fc8d-5ac2-4027-bf28-8c33c64fc4a0 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:45.327819 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb7fc8d-5ac2-4027-bf28-8c33c64fc4a0 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:45.329624 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb7fc8d-5ac2-4027-bf28-8c33c64fc4a0 None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:45.345202 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application [None req-2bb7fc8d-5ac2-4027-bf28-8c33c64fc4a0 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Oct 10 11:46:45.345202 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:46:45.345202 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:46:45.345202 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:46:45.345202 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:46:45.345202 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:46:45.345202 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:46:45.345202 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:46:45.345202 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:46:45.345202 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:46:45.345202 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:46:45.345202 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:46:45.345202 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 10 11:46:45.345202 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 10 11:46:45.345202 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 10 11:46:45.345202 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 10 11:46:45.348050 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:46:45.348050 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:46:45.348050 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 10 11:46:45.348050 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 10 11:46:45.348050 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 10 11:46:45.348050 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 10 11:46:45.348050 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:46:45.348050 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:46:45.348050 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:46:45.348050 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:46:45.348050 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:46:45.348050 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application with Lock( Oct 10 11:46:45.348050 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:46:45.348050 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:46:45.348050 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:46:45.348050 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:46:45.348050 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:46:45.348050 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:46:45.349836 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:46:45.349836 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:46:45.349836 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 10 11:46:45.349836 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 10 11:46:45.349836 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 10 11:46:45.349836 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 10 11:46:45.349836 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 10 11:46:45.349836 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 10 11:46:45.349836 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Oct 10 11:46:45.349836 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Oct 10 11:46:45.349836 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 235/499] 10.222.0.46 () {56 vars in 1102 bytes} [Mon Oct 10 11:46:45 2022] GET /identity/v3/users/neutron => generated 85 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:46:45.349836 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2458c3-6ad2-4b5d-9bbf-f4a6b6ea0a3c None None] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:45.349836 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2458c3-6ad2-4b5d-9bbf-f4a6b6ea0a3c None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:45.351773 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2458c3-6ad2-4b5d-9bbf-f4a6b6ea0a3c None None] PATH_INFO: `/` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:45.351773 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 236/500] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 10 11:46:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:46:45.351773 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2458c3-6ad2-4b5d-9bbf-f4a6b6ea0a3c None None] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:45.399546 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a3301f-32e7-4dcf-bbf4-f63930154ce5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:45.401110 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a3301f-32e7-4dcf-bbf4-f63930154ce5 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:45.417818 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:45.448309 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:45.500498 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2458c3-6ad2-4b5d-9bbf-f4a6b6ea0a3c None None] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:45.545273 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 265/501] 10.222.0.46 () {58 vars in 1282 bytes} [Mon Oct 10 11:46:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 521 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Oct 10 11:46:45.546695 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a3301f-32e7-4dcf-bbf4-f63930154ce5 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:45.559122 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be819a38-39b4-4e7a-9ba0-eb02faf5eaed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:45.559772 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be819a38-39b4-4e7a-9ba0-eb02faf5eaed None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:45.560049 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be819a38-39b4-4e7a-9ba0-eb02faf5eaed None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:45.560974 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be819a38-39b4-4e7a-9ba0-eb02faf5eaed None admin] PATH_INFO: `/v3/roles/service` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:45.574614 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be819a38-39b4-4e7a-9ba0-eb02faf5eaed None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 10 11:46:45.584528 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application [None req-be819a38-39b4-4e7a-9ba0-eb02faf5eaed None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 10 11:46:45.584528 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:46:45.584528 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:46:45.584528 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:46:45.584528 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:46:45.584528 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:46:45.584528 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:46:45.584528 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:46:45.584528 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:46:45.584528 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:46:45.584528 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:46:45.584528 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:46:45.584528 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 10 11:46:45.584528 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 10 11:46:45.584528 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 10 11:46:45.584528 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 10 11:46:45.586266 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 10 11:46:45.586266 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 10 11:46:45.586266 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 10 11:46:45.586266 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 10 11:46:45.586266 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:46:45.586266 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:46:45.586266 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:46:45.586266 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:46:45.586266 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:46:45.586266 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:46:45.586266 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:46:45.586266 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application with Lock( Oct 10 11:46:45.586266 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:46:45.586266 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:46:45.586266 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:46:45.586266 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:46:45.587290 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:46:45.587290 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:46:45.587290 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:46:45.587290 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:46:45.587290 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 10 11:46:45.587290 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 10 11:46:45.587290 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 10 11:46:45.587290 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 10 11:46:45.587290 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 10 11:46:45.587290 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 10 11:46:45.587290 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 10 11:46:45.587290 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Oct 10 11:46:45.587290 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 237/502] 10.222.0.46 () {56 vars in 1102 bytes} [Mon Oct 10 11:46:45 2022] GET /identity/v3/roles/service => generated 85 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:46:45.589093 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58874f0-3a3c-4ef8-a61d-cc5e3ffdeebd None None] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:45.589577 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58874f0-3a3c-4ef8-a61d-cc5e3ffdeebd None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:45.590061 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58874f0-3a3c-4ef8-a61d-cc5e3ffdeebd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:45.612914 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.auth.core [None req-c58874f0-3a3c-4ef8-a61d-cc5e3ffdeebd None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:46:45.614375 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-c58874f0-3a3c-4ef8-a61d-cc5e3ffdeebd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:46:45.681816 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a3301f-32e7-4dcf-bbf4-f63930154ce5 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:45.695536 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 238/503] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 10 11:46:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 10 11:46:45.696463 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58874f0-3a3c-4ef8-a61d-cc5e3ffdeebd None None] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:45.748629 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:45.758078 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:45.761890 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044064c1-7a3a-4831-85bb-579c3fde1832 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:45.762973 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044064c1-7a3a-4831-85bb-579c3fde1832 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:45.763455 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044064c1-7a3a-4831-85bb-579c3fde1832 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:45.763931 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044064c1-7a3a-4831-85bb-579c3fde1832 None admin] PATH_INFO: `/v3/users` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:45.766869 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044064c1-7a3a-4831-85bb-579c3fde1832 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:46:45.766869 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044064c1-7a3a-4831-85bb-579c3fde1832 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:45.768588 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044064c1-7a3a-4831-85bb-579c3fde1832 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:45.805310 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 266/504] 10.222.0.46 () {56 vars in 1139 bytes} [Mon Oct 10 11:46:45 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:46:45.806839 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044064c1-7a3a-4831-85bb-579c3fde1832 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:45.837563 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58874f0-3a3c-4ef8-a61d-cc5e3ffdeebd None None] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:45.882693 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93de4dfd-c965-4af2-a15e-3ac45464e1cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:45.883705 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93de4dfd-c965-4af2-a15e-3ac45464e1cd None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:45.884191 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93de4dfd-c965-4af2-a15e-3ac45464e1cd None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:45.884769 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93de4dfd-c965-4af2-a15e-3ac45464e1cd None admin] PATH_INFO: `/v3/roles/service` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:45.902791 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93de4dfd-c965-4af2-a15e-3ac45464e1cd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 10 11:46:45.913470 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application [None req-93de4dfd-c965-4af2-a15e-3ac45464e1cd None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 10 11:46:45.913470 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:46:45.913470 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:46:45.913470 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:46:45.913470 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:46:45.913470 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:46:45.913470 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:46:45.913470 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:46:45.913470 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:46:45.913470 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:46:45.913470 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:46:45.913470 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:46:45.913470 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 10 11:46:45.913470 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 10 11:46:45.913470 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 10 11:46:45.913470 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 10 11:46:45.914467 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 10 11:46:45.914467 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 10 11:46:45.914467 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 10 11:46:45.914467 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 10 11:46:45.914467 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:46:45.914467 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:46:45.914467 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:46:45.914467 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:46:45.914467 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:46:45.914467 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:46:45.914467 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:46:45.914467 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application with Lock( Oct 10 11:46:45.914467 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:46:45.914467 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:46:45.914467 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:46:45.914467 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:46:45.915345 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:46:45.915345 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:46:45.915345 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:46:45.915345 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:46:45.915345 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 10 11:46:45.915345 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 10 11:46:45.915345 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 10 11:46:45.915345 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 10 11:46:45.915345 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 10 11:46:45.915345 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 10 11:46:45.915345 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 10 11:46:45.915345 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Oct 10 11:46:45.916497 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 239/505] 10.222.0.46 () {56 vars in 1102 bytes} [Mon Oct 10 11:46:45 2022] GET /identity/v3/roles/service => generated 85 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:46:45.918173 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93de4dfd-c965-4af2-a15e-3ac45464e1cd None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:45.992569 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044064c1-7a3a-4831-85bb-579c3fde1832 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:46.044137 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169898c0-da02-426f-aedd-b2fe00a94a4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:46.045283 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169898c0-da02-426f-aedd-b2fe00a94a4a None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:46.045819 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169898c0-da02-426f-aedd-b2fe00a94a4a None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:46.046318 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169898c0-da02-426f-aedd-b2fe00a94a4a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:46.060471 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169898c0-da02-426f-aedd-b2fe00a94a4a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:46.062616 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169898c0-da02-426f-aedd-b2fe00a94a4a None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:46.078969 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93de4dfd-c965-4af2-a15e-3ac45464e1cd None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:46.083076 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:46.089437 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application [None req-169898c0-da02-426f-aedd-b2fe00a94a4a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:46:46.089437 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:46:46.089437 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:46:46.089437 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:46:46.089437 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:46:46.089437 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:46:46.089437 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:46:46.089437 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:46:46.089437 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:46:46.089437 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:46:46.089437 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:46:46.089437 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:46:46.089437 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 10 11:46:46.089437 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 10 11:46:46.089437 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 10 11:46:46.089437 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 10 11:46:46.091513 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:46:46.091513 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:46:46.091513 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:46:46.091513 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:46:46.091513 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:46:46.091513 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:46:46.091513 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:46:46.091513 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application with Lock( Oct 10 11:46:46.091513 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:46:46.091513 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:46:46.091513 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:46:46.091513 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:46:46.091513 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:46:46.091513 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:46:46.091513 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:46:46.091513 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:46:46.091513 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 10 11:46:46.093823 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 10 11:46:46.093823 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:46:46.093823 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Oct 10 11:46:46.093823 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 267/506] 10.222.0.46 () {56 vars in 1108 bytes} [Mon Oct 10 11:46:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:46:46.094227 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169898c0-da02-426f-aedd-b2fe00a94a4a None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:46.131902 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96aa669-07bd-434b-a713-f6681ff13c52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:46.132696 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96aa669-07bd-434b-a713-f6681ff13c52 None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:46.133042 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96aa669-07bd-434b-a713-f6681ff13c52 None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:46.133383 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96aa669-07bd-434b-a713-f6681ff13c52 None admin] PATH_INFO: `/v3/roles` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:46.134220 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96aa669-07bd-434b-a713-f6681ff13c52 None admin] RBAC: Adding query filter params (name=service) {{(pid=81676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:46:46.134704 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96aa669-07bd-434b-a713-f6681ff13c52 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:46.136460 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96aa669-07bd-434b-a713-f6681ff13c52 None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:46.147274 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 240/507] 10.222.0.46 () {56 vars in 1103 bytes} [Mon Oct 10 11:46:45 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:46:46.148660 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:46.148850 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96aa669-07bd-434b-a713-f6681ff13c52 None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:46.222282 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169898c0-da02-426f-aedd-b2fe00a94a4a None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:46.265469 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813a3dea-9af1-4621-b372-032172ce476c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:46.266260 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813a3dea-9af1-4621-b372-032172ce476c None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:46.266599 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813a3dea-9af1-4621-b372-032172ce476c None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:46.266970 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813a3dea-9af1-4621-b372-032172ce476c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:46.278372 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96aa669-07bd-434b-a713-f6681ff13c52 None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:46.328226 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb69f6ba-1378-40a9-b615-1b88f81252b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:46.329108 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb69f6ba-1378-40a9-b615-1b88f81252b2 None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:46.329561 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb69f6ba-1378-40a9-b615-1b88f81252b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:46.330015 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb69f6ba-1378-40a9-b615-1b88f81252b2 None admin] PATH_INFO: `/v3/roles` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:46.331143 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb69f6ba-1378-40a9-b615-1b88f81252b2 None admin] RBAC: Adding query filter params (name=service) {{(pid=81676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:46:46.331647 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb69f6ba-1378-40a9-b615-1b88f81252b2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:46.334049 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb69f6ba-1378-40a9-b615-1b88f81252b2 None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:46.348393 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 241/508] 10.222.0.46 () {56 vars in 1103 bytes} [Mon Oct 10 11:46:46 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:46:46.349337 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813a3dea-9af1-4621-b372-032172ce476c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:46.349650 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb69f6ba-1378-40a9-b615-1b88f81252b2 None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:46.350325 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813a3dea-9af1-4621-b372-032172ce476c None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:46.393830 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:46.400144 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:46.465659 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 268/509] 10.222.0.46 () {58 vars in 1282 bytes} [Mon Oct 10 11:46:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 10 11:46:46.466580 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813a3dea-9af1-4621-b372-032172ce476c None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:46.494217 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb69f6ba-1378-40a9-b615-1b88f81252b2 None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:46.532510 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd974bf-6ea2-457b-86ef-c574bd0fec4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:46.533147 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd974bf-6ea2-457b-86ef-c574bd0fec4f None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:46.533425 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd974bf-6ea2-457b-86ef-c574bd0fec4f None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:46.533696 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd974bf-6ea2-457b-86ef-c574bd0fec4f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:46.544772 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd974bf-6ea2-457b-86ef-c574bd0fec4f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:46.546818 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd974bf-6ea2-457b-86ef-c574bd0fec4f None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:46.563299 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application [None req-9fd974bf-6ea2-457b-86ef-c574bd0fec4f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:46:46.563299 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:46:46.563299 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:46:46.563299 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:46:46.563299 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:46:46.563299 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:46:46.563299 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:46:46.563299 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:46:46.563299 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:46:46.563299 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:46:46.563299 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:46:46.563299 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:46:46.563299 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 10 11:46:46.563299 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 10 11:46:46.563299 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 10 11:46:46.563299 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 10 11:46:46.565158 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:46:46.565158 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:46:46.565158 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:46:46.565158 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:46:46.565158 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:46:46.565158 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:46:46.565158 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:46:46.565158 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application with Lock( Oct 10 11:46:46.565158 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:46:46.565158 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:46:46.565158 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:46:46.565158 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:46:46.565158 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:46:46.565158 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:46:46.565158 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:46:46.565158 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:46:46.565158 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 10 11:46:46.566820 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 10 11:46:46.566820 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:46:46.566820 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Oct 10 11:46:46.566820 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 242/510] 10.222.0.46 () {56 vars in 1108 bytes} [Mon Oct 10 11:46:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:46:46.566820 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd974bf-6ea2-457b-86ef-c574bd0fec4f None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:46.605380 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813a3dea-9af1-4621-b372-032172ce476c None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:46.651940 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:46.660105 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b7658d-6614-482a-8071-07f9d0a34926 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:46.660718 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b7658d-6614-482a-8071-07f9d0a34926 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:46.662110 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b7658d-6614-482a-8071-07f9d0a34926 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:46.662110 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b7658d-6614-482a-8071-07f9d0a34926 None admin] PATH_INFO: `/v3/domains` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:46.662110 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b7658d-6614-482a-8071-07f9d0a34926 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:46:46.662453 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b7658d-6614-482a-8071-07f9d0a34926 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:46.663773 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b7658d-6614-482a-8071-07f9d0a34926 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:46.673795 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:46.674655 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd974bf-6ea2-457b-86ef-c574bd0fec4f None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:46.678099 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 269/511] 10.222.0.46 () {56 vars in 1109 bytes} [Mon Oct 10 11:46:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:46:46.679511 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b7658d-6614-482a-8071-07f9d0a34926 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:46.726890 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7adf2c6d-72dd-49aa-85e6-0ce38a8dc5b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:46.727586 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adf2c6d-72dd-49aa-85e6-0ce38a8dc5b6 None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:46.727931 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adf2c6d-72dd-49aa-85e6-0ce38a8dc5b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:46.728269 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adf2c6d-72dd-49aa-85e6-0ce38a8dc5b6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:46.740359 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7adf2c6d-72dd-49aa-85e6-0ce38a8dc5b6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:46.742105 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7adf2c6d-72dd-49aa-85e6-0ce38a8dc5b6 None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:46.759527 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application [None req-7adf2c6d-72dd-49aa-85e6-0ce38a8dc5b6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:46:46.759527 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:46:46.759527 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:46:46.759527 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:46:46.759527 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:46:46.759527 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:46:46.759527 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:46:46.759527 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:46:46.759527 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:46:46.759527 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:46:46.759527 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:46:46.759527 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:46:46.759527 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 10 11:46:46.759527 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 10 11:46:46.759527 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 10 11:46:46.759527 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 10 11:46:46.761386 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:46:46.761386 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:46:46.761386 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:46:46.761386 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:46:46.761386 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:46:46.761386 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:46:46.761386 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:46:46.761386 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application with Lock( Oct 10 11:46:46.761386 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:46:46.761386 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:46:46.761386 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:46:46.761386 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:46:46.761386 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:46:46.761386 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:46:46.761386 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:46:46.761386 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:46:46.761386 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 10 11:46:46.762369 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 10 11:46:46.762369 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:46:46.762369 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Oct 10 11:46:46.762369 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 243/512] 10.222.0.46 () {56 vars in 1108 bytes} [Mon Oct 10 11:46:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:46:46.762369 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7adf2c6d-72dd-49aa-85e6-0ce38a8dc5b6 None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:46.827762 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b7658d-6614-482a-8071-07f9d0a34926 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:46.870166 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf5c281-17ab-4da5-afbe-dde7ea48e6ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:46.870891 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf5c281-17ab-4da5-afbe-dde7ea48e6ca None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:46.871535 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf5c281-17ab-4da5-afbe-dde7ea48e6ca None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:46.871895 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf5c281-17ab-4da5-afbe-dde7ea48e6ca None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:46.884350 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf5c281-17ab-4da5-afbe-dde7ea48e6ca None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:46.886334 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf5c281-17ab-4da5-afbe-dde7ea48e6ca None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:46.894308 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7adf2c6d-72dd-49aa-85e6-0ce38a8dc5b6 None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:46.902889 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application [None req-bdf5c281-17ab-4da5-afbe-dde7ea48e6ca None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:46:46.902889 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:46:46.902889 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:46:46.902889 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:46:46.902889 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:46:46.902889 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:46:46.902889 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:46:46.902889 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:46:46.902889 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:46:46.902889 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:46:46.902889 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:46:46.902889 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:46:46.902889 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 10 11:46:46.902889 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 10 11:46:46.902889 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 10 11:46:46.902889 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 10 11:46:46.904800 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:46:46.904800 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:46:46.904800 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:46:46.904800 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:46:46.904800 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:46:46.904800 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:46:46.904800 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:46:46.904800 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application with Lock( Oct 10 11:46:46.904800 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:46:46.904800 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:46:46.904800 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:46:46.904800 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:46:46.904800 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:46:46.904800 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:46:46.904800 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:46:46.904800 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:46:46.904800 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 10 11:46:46.906174 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 10 11:46:46.906174 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:46:46.906174 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Oct 10 11:46:46.906174 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 270/513] 10.222.0.46 () {56 vars in 1108 bytes} [Mon Oct 10 11:46:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:46:46.906439 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf5c281-17ab-4da5-afbe-dde7ea48e6ca None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:46.937280 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:46.944744 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea393f8-98b8-4318-822b-10316ec11fe5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:46.945734 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea393f8-98b8-4318-822b-10316ec11fe5 None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:46.946469 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea393f8-98b8-4318-822b-10316ec11fe5 None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:46.946611 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea393f8-98b8-4318-822b-10316ec11fe5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:46.965311 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea393f8-98b8-4318-822b-10316ec11fe5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:46.968036 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea393f8-98b8-4318-822b-10316ec11fe5 None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:46.988073 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application [None req-4ea393f8-98b8-4318-822b-10316ec11fe5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:46:46.988073 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:46:46.988073 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:46:46.988073 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:46:46.988073 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:46:46.988073 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:46:46.988073 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:46:46.988073 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:46:46.988073 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:46:46.988073 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:46:46.988073 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:46:46.988073 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:46:46.988073 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 10 11:46:46.988073 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 10 11:46:46.988073 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 10 11:46:46.988073 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 10 11:46:46.990099 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:46:46.990099 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:46:46.990099 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:46:46.990099 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:46:46.990099 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:46:46.990099 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:46:46.990099 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:46:46.990099 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application with Lock( Oct 10 11:46:46.990099 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:46:46.990099 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:46:46.990099 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:46:46.990099 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:46:46.990099 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:46:46.990099 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:46:46.990099 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:46:46.990099 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:46:46.990099 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 10 11:46:46.992879 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 10 11:46:46.992879 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:46:46.992879 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Oct 10 11:46:46.993360 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:46.993542 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 244/514] 10.222.0.46 () {56 vars in 1108 bytes} [Mon Oct 10 11:46:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:46:46.993542 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea393f8-98b8-4318-822b-10316ec11fe5 None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:47.052427 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf5c281-17ab-4da5-afbe-dde7ea48e6ca None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:47.091546 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a95c5c-6fd9-4e2e-a0c2-5364bcc8656f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:47.092172 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a95c5c-6fd9-4e2e-a0c2-5364bcc8656f None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:47.092508 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a95c5c-6fd9-4e2e-a0c2-5364bcc8656f None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:47.092786 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a95c5c-6fd9-4e2e-a0c2-5364bcc8656f None admin] PATH_INFO: `/v3/domains` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:47.093453 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a95c5c-6fd9-4e2e-a0c2-5364bcc8656f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:46:47.093752 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a95c5c-6fd9-4e2e-a0c2-5364bcc8656f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:47.095165 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a95c5c-6fd9-4e2e-a0c2-5364bcc8656f None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:47.108953 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 271/515] 10.222.0.46 () {56 vars in 1109 bytes} [Mon Oct 10 11:46:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:46:47.109839 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a95c5c-6fd9-4e2e-a0c2-5364bcc8656f None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:47.132124 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea393f8-98b8-4318-822b-10316ec11fe5 None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:47.181082 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af39401f-648a-4682-87a6-0e05b3ad47bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:47.181835 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af39401f-648a-4682-87a6-0e05b3ad47bf None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:47.182116 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af39401f-648a-4682-87a6-0e05b3ad47bf None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:47.182386 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af39401f-648a-4682-87a6-0e05b3ad47bf None admin] PATH_INFO: `/v3/projects/service` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:47.189723 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af39401f-648a-4682-87a6-0e05b3ad47bf None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:47.191254 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af39401f-648a-4682-87a6-0e05b3ad47bf None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:47.204560 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application [None req-af39401f-648a-4682-87a6-0e05b3ad47bf None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 10 11:46:47.204560 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:46:47.204560 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:46:47.204560 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:46:47.204560 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:46:47.204560 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:46:47.204560 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:46:47.204560 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:46:47.204560 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:46:47.204560 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:46:47.204560 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:46:47.204560 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:46:47.204560 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 10 11:46:47.204560 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 10 11:46:47.204560 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 10 11:46:47.205521 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 10 11:46:47.205521 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:46:47.205521 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:46:47.205521 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:46:47.205521 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:46:47.205521 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:46:47.205521 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:46:47.205521 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:46:47.205521 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application with Lock( Oct 10 11:46:47.205521 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:46:47.205521 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:46:47.205521 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:46:47.205521 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:46:47.205521 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:46:47.205521 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:46:47.205521 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:46:47.205521 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:46:47.207113 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 10 11:46:47.207113 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 10 11:46:47.207113 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 10 11:46:47.207113 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 10 11:46:47.207113 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 10 11:46:47.207113 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 10 11:46:47.207113 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 10 11:46:47.207113 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Oct 10 11:46:47.207113 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 245/516] 10.222.0.46 () {56 vars in 1111 bytes} [Mon Oct 10 11:46:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:46:47.207113 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af39401f-648a-4682-87a6-0e05b3ad47bf None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:47.234970 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a95c5c-6fd9-4e2e-a0c2-5364bcc8656f None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:47.251603 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:47.273686 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22bc17a-4f41-45ed-8c6e-a5908a1fed49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:47.274298 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22bc17a-4f41-45ed-8c6e-a5908a1fed49 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:47.274577 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22bc17a-4f41-45ed-8c6e-a5908a1fed49 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:47.274880 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22bc17a-4f41-45ed-8c6e-a5908a1fed49 None admin] PATH_INFO: `/v3/domains` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:47.275524 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22bc17a-4f41-45ed-8c6e-a5908a1fed49 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:46:47.275827 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22bc17a-4f41-45ed-8c6e-a5908a1fed49 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:47.277207 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22bc17a-4f41-45ed-8c6e-a5908a1fed49 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:47.289408 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:47.289928 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 272/517] 10.222.0.46 () {56 vars in 1109 bytes} [Mon Oct 10 11:46:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:46:47.291594 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22bc17a-4f41-45ed-8c6e-a5908a1fed49 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:47.358224 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af39401f-648a-4682-87a6-0e05b3ad47bf None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:47.409176 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ebcc2f-b9a6-46f1-9a5b-4466f4cca968 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:47.410078 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ebcc2f-b9a6-46f1-9a5b-4466f4cca968 None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:47.410567 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ebcc2f-b9a6-46f1-9a5b-4466f4cca968 None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:47.411064 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ebcc2f-b9a6-46f1-9a5b-4466f4cca968 None admin] PATH_INFO: `/v3/domains` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:47.412211 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ebcc2f-b9a6-46f1-9a5b-4466f4cca968 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:46:47.412749 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ebcc2f-b9a6-46f1-9a5b-4466f4cca968 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:47.415531 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ebcc2f-b9a6-46f1-9a5b-4466f4cca968 None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:47.418337 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22bc17a-4f41-45ed-8c6e-a5908a1fed49 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:47.436185 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 246/518] 10.222.0.46 () {56 vars in 1109 bytes} [Mon Oct 10 11:46:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:46:47.437877 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ebcc2f-b9a6-46f1-9a5b-4466f4cca968 None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:47.457112 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bd1561-0c78-43e2-a2e4-cb40ca8ea367 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:47.458332 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bd1561-0c78-43e2-a2e4-cb40ca8ea367 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:47.458613 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bd1561-0c78-43e2-a2e4-cb40ca8ea367 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:47.458925 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bd1561-0c78-43e2-a2e4-cb40ca8ea367 None admin] PATH_INFO: `/v3/domains` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:47.459607 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56bd1561-0c78-43e2-a2e4-cb40ca8ea367 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:46:47.459911 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56bd1561-0c78-43e2-a2e4-cb40ca8ea367 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:47.461908 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56bd1561-0c78-43e2-a2e4-cb40ca8ea367 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:47.479247 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 273/519] 10.222.0.46 () {56 vars in 1109 bytes} [Mon Oct 10 11:46:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:46:47.480569 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bd1561-0c78-43e2-a2e4-cb40ca8ea367 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:47.519468 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:47.600376 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ebcc2f-b9a6-46f1-9a5b-4466f4cca968 None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:47.603003 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:47.609049 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bd1561-0c78-43e2-a2e4-cb40ca8ea367 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:47.641107 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88031530-903f-4916-bd27-808234362c47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:47.641724 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88031530-903f-4916-bd27-808234362c47 None admin] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:47.642003 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88031530-903f-4916-bd27-808234362c47 None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:47.642273 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88031530-903f-4916-bd27-808234362c47 None admin] PATH_INFO: `/v3/users` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:47.643003 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88031530-903f-4916-bd27-808234362c47 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:47.644454 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88031530-903f-4916-bd27-808234362c47 None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:47.650814 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa79562-a93b-400c-b41f-3a69dac6608b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:47.651492 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa79562-a93b-400c-b41f-3a69dac6608b None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:47.651815 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa79562-a93b-400c-b41f-3a69dac6608b None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:47.652135 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa79562-a93b-400c-b41f-3a69dac6608b None admin] PATH_INFO: `/v3/projects` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:47.652983 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa79562-a93b-400c-b41f-3a69dac6608b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:46:47.653331 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa79562-a93b-400c-b41f-3a69dac6608b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:47.655529 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa79562-a93b-400c-b41f-3a69dac6608b None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:47.676168 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 274/520] 10.222.0.46 () {56 vars in 1148 bytes} [Mon Oct 10 11:46:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:46:47.677457 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa79562-a93b-400c-b41f-3a69dac6608b None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:47.757454 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 247/521] 10.222.0.46 () {60 vars in 1132 bytes} [Mon Oct 10 11:46:47 2022] POST /identity/v3/users => generated 247 bytes in 321 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 11:46:47.759057 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88031530-903f-4916-bd27-808234362c47 None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:47.793059 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:47.808682 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa79562-a93b-400c-b41f-3a69dac6608b None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:47.872842 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc684ab-be26-4675-a591-c9e6060aebbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:47.873977 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc684ab-be26-4675-a591-c9e6060aebbc None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:47.874531 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc684ab-be26-4675-a591-c9e6060aebbc None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:47.874996 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc684ab-be26-4675-a591-c9e6060aebbc None admin] PATH_INFO: `/v3/users/nova` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:47.889126 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc684ab-be26-4675-a591-c9e6060aebbc None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:47.891833 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc684ab-be26-4675-a591-c9e6060aebbc None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:47.915774 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application [None req-ddc684ab-be26-4675-a591-c9e6060aebbc None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 10 11:46:47.915774 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:46:47.915774 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:46:47.915774 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:46:47.915774 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:46:47.915774 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:46:47.915774 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:46:47.915774 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:46:47.915774 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:46:47.915774 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:46:47.915774 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:46:47.915774 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:46:47.915774 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 10 11:46:47.915774 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 10 11:46:47.915774 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 10 11:46:47.915774 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 10 11:46:47.917736 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:46:47.917736 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:46:47.917736 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 10 11:46:47.917736 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 10 11:46:47.917736 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 10 11:46:47.917736 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 10 11:46:47.917736 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:46:47.917736 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:46:47.917736 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:46:47.917736 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:46:47.917736 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:46:47.917736 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application with Lock( Oct 10 11:46:47.917736 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:46:47.917736 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:46:47.917736 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:46:47.917736 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:46:47.917736 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:46:47.917736 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:46:47.920169 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:46:47.920169 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:46:47.920169 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 10 11:46:47.920169 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 10 11:46:47.920169 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 10 11:46:47.920169 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 10 11:46:47.920169 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 10 11:46:47.920169 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 10 11:46:47.920169 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 10 11:46:47.920169 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Oct 10 11:46:47.920169 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 275/522] 10.222.0.46 () {56 vars in 1093 bytes} [Mon Oct 10 11:46:47 2022] GET /identity/v3/users/nova => generated 82 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:46:47.922393 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc684ab-be26-4675-a591-c9e6060aebbc None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:47.954577 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88031530-903f-4916-bd27-808234362c47 None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:48.013616 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c504b7-be1a-4e45-8b3b-8f4657512289 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:48.015024 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c504b7-be1a-4e45-8b3b-8f4657512289 None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:48.015502 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c504b7-be1a-4e45-8b3b-8f4657512289 None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:48.015913 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c504b7-be1a-4e45-8b3b-8f4657512289 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:48.026259 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c504b7-be1a-4e45-8b3b-8f4657512289 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:48.028979 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c504b7-be1a-4e45-8b3b-8f4657512289 None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:48.049447 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application [None req-86c504b7-be1a-4e45-8b3b-8f4657512289 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Oct 10 11:46:48.049447 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:46:48.049447 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:46:48.049447 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:46:48.049447 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:46:48.049447 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:46:48.049447 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:46:48.049447 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:46:48.049447 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:46:48.049447 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:46:48.049447 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:46:48.049447 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:46:48.049447 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 10 11:46:48.049447 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 10 11:46:48.049447 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 10 11:46:48.049447 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 10 11:46:48.050498 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:46:48.050498 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:46:48.050498 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 10 11:46:48.050498 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 10 11:46:48.050498 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 10 11:46:48.050498 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 10 11:46:48.050498 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:46:48.050498 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:46:48.050498 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:46:48.050498 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:46:48.050498 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:46:48.050498 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application with Lock( Oct 10 11:46:48.050498 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:46:48.050498 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:46:48.050498 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:46:48.050498 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:46:48.050498 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:46:48.050498 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:46:48.051462 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:46:48.051462 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:46:48.051462 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 10 11:46:48.051462 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 10 11:46:48.051462 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 10 11:46:48.051462 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 10 11:46:48.051462 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 10 11:46:48.051462 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 10 11:46:48.051462 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Oct 10 11:46:48.051462 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Oct 10 11:46:48.051462 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 248/523] 10.222.0.46 () {56 vars in 1099 bytes} [Mon Oct 10 11:46:47 2022] GET /identity/v3/users/glance => generated 84 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:46:48.052725 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c504b7-be1a-4e45-8b3b-8f4657512289 None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:48.078261 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc684ab-be26-4675-a591-c9e6060aebbc None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:48.083369 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:48.128061 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03365a08-27d9-41ff-acf8-efadc003e5d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:48.129528 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03365a08-27d9-41ff-acf8-efadc003e5d2 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:48.130013 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03365a08-27d9-41ff-acf8-efadc003e5d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:48.130469 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03365a08-27d9-41ff-acf8-efadc003e5d2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:48.144810 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03365a08-27d9-41ff-acf8-efadc003e5d2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:48.146291 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03365a08-27d9-41ff-acf8-efadc003e5d2 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:48.163122 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application [None req-03365a08-27d9-41ff-acf8-efadc003e5d2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 10 11:46:48.163122 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:46:48.163122 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:46:48.163122 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:46:48.163122 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:46:48.163122 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:46:48.163122 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:46:48.163122 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:46:48.163122 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:46:48.163122 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:46:48.163122 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:46:48.163122 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:46:48.163122 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 10 11:46:48.163122 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 10 11:46:48.163122 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 10 11:46:48.164805 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 10 11:46:48.164805 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:46:48.164805 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:46:48.164805 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:46:48.164805 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:46:48.164805 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:46:48.164805 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:46:48.164805 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:46:48.164805 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application with Lock( Oct 10 11:46:48.164805 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:46:48.164805 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:46:48.164805 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:46:48.164805 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:46:48.164805 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:46:48.164805 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:46:48.164805 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:46:48.164805 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:46:48.167244 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 10 11:46:48.167244 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 10 11:46:48.167244 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 10 11:46:48.167244 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 10 11:46:48.167244 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 10 11:46:48.167244 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 10 11:46:48.167244 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 10 11:46:48.167244 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Oct 10 11:46:48.167244 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 276/524] 10.222.0.46 () {56 vars in 1111 bytes} [Mon Oct 10 11:46:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:46:48.167244 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03365a08-27d9-41ff-acf8-efadc003e5d2 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:48.185805 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c504b7-be1a-4e45-8b3b-8f4657512289 None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:48.188350 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:48.241446 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79eb362b-7603-4d9e-8dd8-14eff95e84f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:48.242323 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79eb362b-7603-4d9e-8dd8-14eff95e84f4 None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:48.242776 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79eb362b-7603-4d9e-8dd8-14eff95e84f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:48.243090 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79eb362b-7603-4d9e-8dd8-14eff95e84f4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:48.244732 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79eb362b-7603-4d9e-8dd8-14eff95e84f4 None admin] RBAC: Adding query filter params (role.id=3c35be38787740f4834c728a2ce73b6b, scope.project.id=4ad3772f7b6a47c28b7ba37a9316a3c1, user.id=d3ae093470484b93b23dcfae1657df9e) {{(pid=81676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:46:48.244732 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79eb362b-7603-4d9e-8dd8-14eff95e84f4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:48.247361 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79eb362b-7603-4d9e-8dd8-14eff95e84f4 None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:48.266706 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 249/525] 10.222.0.46 () {56 vars in 1374 bytes} [Mon Oct 10 11:46:48 2022] GET /identity/v3/role_assignments?user.id=d3ae093470484b93b23dcfae1657df9e&scope.project.id=4ad3772f7b6a47c28b7ba37a9316a3c1&role.id=3c35be38787740f4834c728a2ce73b6b => generated 259 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:46:48.266706 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79eb362b-7603-4d9e-8dd8-14eff95e84f4 None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:48.332308 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03365a08-27d9-41ff-acf8-efadc003e5d2 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:48.397982 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a279e51-baf6-41c0-8e57-c9cad8f8d8e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:48.398953 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a279e51-baf6-41c0-8e57-c9cad8f8d8e5 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:48.399386 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a279e51-baf6-41c0-8e57-c9cad8f8d8e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:48.399981 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a279e51-baf6-41c0-8e57-c9cad8f8d8e5 None admin] PATH_INFO: `/v3/users` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:48.401513 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a279e51-baf6-41c0-8e57-c9cad8f8d8e5 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:46:48.402161 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a279e51-baf6-41c0-8e57-c9cad8f8d8e5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:48.404144 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a279e51-baf6-41c0-8e57-c9cad8f8d8e5 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:48.411238 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:48.429635 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 277/526] 10.222.0.46 () {56 vars in 1133 bytes} [Mon Oct 10 11:46:48 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:46:48.430936 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a279e51-baf6-41c0-8e57-c9cad8f8d8e5 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:48.437349 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79eb362b-7603-4d9e-8dd8-14eff95e84f4 None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:48.511003 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf1692a-e132-4259-9687-12805b390e98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:48.511003 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf1692a-e132-4259-9687-12805b390e98 None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:48.511003 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf1692a-e132-4259-9687-12805b390e98 None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:48.511003 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf1692a-e132-4259-9687-12805b390e98 None admin] PATH_INFO: `/v3/users` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:48.511381 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf1692a-e132-4259-9687-12805b390e98 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:46:48.511381 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf1692a-e132-4259-9687-12805b390e98 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:48.511381 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf1692a-e132-4259-9687-12805b390e98 None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:48.550733 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:48.558789 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 250/527] 10.222.0.46 () {56 vars in 1137 bytes} [Mon Oct 10 11:46:48 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:46:48.560267 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf1692a-e132-4259-9687-12805b390e98 None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:48.596434 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a279e51-baf6-41c0-8e57-c9cad8f8d8e5 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:48.644055 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eebfd5a-468d-4c30-8049-ba3f8b091743 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:48.645029 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eebfd5a-468d-4c30-8049-ba3f8b091743 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:48.645160 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eebfd5a-468d-4c30-8049-ba3f8b091743 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:48.645537 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eebfd5a-468d-4c30-8049-ba3f8b091743 None admin] PATH_INFO: `/v3/projects` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:48.649109 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eebfd5a-468d-4c30-8049-ba3f8b091743 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:46:48.649109 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eebfd5a-468d-4c30-8049-ba3f8b091743 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:48.649498 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eebfd5a-468d-4c30-8049-ba3f8b091743 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:48.667276 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 278/528] 10.222.0.46 () {56 vars in 1148 bytes} [Mon Oct 10 11:46:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:46:48.668789 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eebfd5a-468d-4c30-8049-ba3f8b091743 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:48.713701 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf1692a-e132-4259-9687-12805b390e98 None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:48.775203 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66198270-e835-4bc6-b575-d57dd8f297fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:48.776232 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66198270-e835-4bc6-b575-d57dd8f297fa None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:48.776640 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66198270-e835-4bc6-b575-d57dd8f297fa None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:48.777086 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66198270-e835-4bc6-b575-d57dd8f297fa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:48.799876 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:48.817912 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eebfd5a-468d-4c30-8049-ba3f8b091743 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:48.873306 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:48.883686 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36126842-98e1-4e6c-b4b4-9f595f44951f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:48.885000 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36126842-98e1-4e6c-b4b4-9f595f44951f None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:48.885349 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36126842-98e1-4e6c-b4b4-9f595f44951f None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:48.885842 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36126842-98e1-4e6c-b4b4-9f595f44951f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:48.887647 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66198270-e835-4bc6-b575-d57dd8f297fa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:48.889398 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66198270-e835-4bc6-b575-d57dd8f297fa None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:48.995304 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36126842-98e1-4e6c-b4b4-9f595f44951f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:48.998185 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36126842-98e1-4e6c-b4b4-9f595f44951f None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:49.057973 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 251/529] 10.222.0.46 () {58 vars in 1282 bytes} [Mon Oct 10 11:46:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 498 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 10 11:46:49.059566 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66198270-e835-4bc6-b575-d57dd8f297fa None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:49.135557 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 279/530] 10.222.0.46 () {58 vars in 1282 bytes} [Mon Oct 10 11:46:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 468 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 10 11:46:49.136670 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36126842-98e1-4e6c-b4b4-9f595f44951f None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:49.147591 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:49.185167 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:49.227927 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66198270-e835-4bc6-b575-d57dd8f297fa None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:49.279243 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e68d422-16e3-4f43-bbdc-d2c82e482c77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:49.280207 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e68d422-16e3-4f43-bbdc-d2c82e482c77 None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:49.280490 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e68d422-16e3-4f43-bbdc-d2c82e482c77 None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:49.280763 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e68d422-16e3-4f43-bbdc-d2c82e482c77 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:49.281512 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e68d422-16e3-4f43-bbdc-d2c82e482c77 None admin] RBAC: Adding query filter params (role.id=3c35be38787740f4834c728a2ce73b6b, scope.project.id=4ad3772f7b6a47c28b7ba37a9316a3c1, user.id=58b1c7c85302421cba823ace4201861f) {{(pid=81676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:46:49.281821 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e68d422-16e3-4f43-bbdc-d2c82e482c77 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:49.284103 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e68d422-16e3-4f43-bbdc-d2c82e482c77 None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:49.298717 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 252/531] 10.222.0.46 () {56 vars in 1374 bytes} [Mon Oct 10 11:46:49 2022] GET /identity/v3/role_assignments?user.id=58b1c7c85302421cba823ace4201861f&scope.project.id=4ad3772f7b6a47c28b7ba37a9316a3c1&role.id=3c35be38787740f4834c728a2ce73b6b => generated 259 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:46:49.298717 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e68d422-16e3-4f43-bbdc-d2c82e482c77 None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:49.321650 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36126842-98e1-4e6c-b4b4-9f595f44951f None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:49.410508 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3a0e4c-4c51-41ec-8c7b-9f2f12016e29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:49.411203 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3a0e4c-4c51-41ec-8c7b-9f2f12016e29 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:49.411483 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3a0e4c-4c51-41ec-8c7b-9f2f12016e29 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:49.411767 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3a0e4c-4c51-41ec-8c7b-9f2f12016e29 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:49.427925 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3a0e4c-4c51-41ec-8c7b-9f2f12016e29 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:49.433802 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3a0e4c-4c51-41ec-8c7b-9f2f12016e29 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:49.450358 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application [None req-4b3a0e4c-4c51-41ec-8c7b-9f2f12016e29 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:46:49.450358 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:46:49.450358 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:46:49.450358 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:46:49.450358 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:46:49.450358 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:46:49.450358 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:46:49.450358 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:46:49.450358 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:46:49.450358 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:46:49.450358 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:46:49.450358 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:46:49.450358 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 10 11:46:49.450358 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 10 11:46:49.450358 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 10 11:46:49.450358 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 10 11:46:49.453097 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:46:49.453097 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:46:49.453097 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:46:49.453097 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:46:49.453097 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:46:49.453097 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:46:49.453097 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:46:49.453097 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application with Lock( Oct 10 11:46:49.453097 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:46:49.453097 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:46:49.453097 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:46:49.453097 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:46:49.453097 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:46:49.453097 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:46:49.453097 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:46:49.453097 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:46:49.453097 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 10 11:46:49.455518 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 10 11:46:49.455518 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:46:49.455518 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Oct 10 11:46:49.455518 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 280/532] 10.222.0.46 () {56 vars in 1108 bytes} [Mon Oct 10 11:46:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 318 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:46:49.460039 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3a0e4c-4c51-41ec-8c7b-9f2f12016e29 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:49.602340 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:49.609999 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3a0e4c-4c51-41ec-8c7b-9f2f12016e29 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:49.636313 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e68d422-16e3-4f43-bbdc-d2c82e482c77 None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:49.652954 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:49.658619 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35ae183-3c95-4677-8d1d-a7514aa741b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:49.660015 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35ae183-3c95-4677-8d1d-a7514aa741b9 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:49.660513 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35ae183-3c95-4677-8d1d-a7514aa741b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:49.661013 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35ae183-3c95-4677-8d1d-a7514aa741b9 None admin] PATH_INFO: `/v3/domains` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:49.662435 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35ae183-3c95-4677-8d1d-a7514aa741b9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:46:49.663219 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35ae183-3c95-4677-8d1d-a7514aa741b9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:49.665052 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35ae183-3c95-4677-8d1d-a7514aa741b9 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:49.678868 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 281/533] 10.222.0.46 () {56 vars in 1109 bytes} [Mon Oct 10 11:46:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:46:49.691839 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35ae183-3c95-4677-8d1d-a7514aa741b9 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:49.736567 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b048d859-ed2c-4e9f-96d0-bdc4ed61e8dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:49.737717 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b048d859-ed2c-4e9f-96d0-bdc4ed61e8dd None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:49.738229 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b048d859-ed2c-4e9f-96d0-bdc4ed61e8dd None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:49.738749 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b048d859-ed2c-4e9f-96d0-bdc4ed61e8dd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:49.779642 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b048d859-ed2c-4e9f-96d0-bdc4ed61e8dd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:49.783046 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b048d859-ed2c-4e9f-96d0-bdc4ed61e8dd None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:49.818780 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application [None req-b048d859-ed2c-4e9f-96d0-bdc4ed61e8dd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:46:49.818780 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:46:49.818780 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:46:49.818780 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:46:49.818780 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:46:49.818780 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:46:49.818780 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:46:49.818780 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:46:49.818780 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:46:49.818780 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:46:49.818780 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:46:49.818780 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:46:49.818780 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 10 11:46:49.818780 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 10 11:46:49.818780 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 10 11:46:49.818780 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 10 11:46:49.819806 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:46:49.819806 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:46:49.819806 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:46:49.819806 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:46:49.819806 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:46:49.819806 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:46:49.819806 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:46:49.819806 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application with Lock( Oct 10 11:46:49.819806 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:46:49.819806 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:46:49.819806 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:46:49.819806 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:46:49.819806 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:46:49.819806 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:46:49.819806 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:46:49.819806 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:46:49.819806 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 10 11:46:49.820687 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 10 11:46:49.820687 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:46:49.820687 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Oct 10 11:46:49.828101 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 253/534] 10.222.0.46 () {56 vars in 1108 bytes} [Mon Oct 10 11:46:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 536 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:46:49.834492 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b048d859-ed2c-4e9f-96d0-bdc4ed61e8dd None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:50.021923 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35ae183-3c95-4677-8d1d-a7514aa741b9 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:50.078067 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b048d859-ed2c-4e9f-96d0-bdc4ed61e8dd None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:50.138886 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c30abf7-b909-4094-a093-9257c46906c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:50.140471 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c30abf7-b909-4094-a093-9257c46906c3 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:50.140932 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c30abf7-b909-4094-a093-9257c46906c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:50.141384 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c30abf7-b909-4094-a093-9257c46906c3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:50.153489 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c30abf7-b909-4094-a093-9257c46906c3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:50.156355 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c30abf7-b909-4094-a093-9257c46906c3 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:50.188673 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application [None req-6c30abf7-b909-4094-a093-9257c46906c3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 10 11:46:50.188673 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:46:50.188673 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:46:50.188673 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:46:50.188673 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:46:50.188673 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:46:50.188673 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:46:50.188673 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:46:50.188673 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:46:50.188673 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:46:50.188673 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:46:50.188673 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:46:50.188673 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 10 11:46:50.188673 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 10 11:46:50.188673 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 10 11:46:50.189645 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 10 11:46:50.189645 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:46:50.189645 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:46:50.189645 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:46:50.189645 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:46:50.189645 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:46:50.189645 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:46:50.189645 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:46:50.189645 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application with Lock( Oct 10 11:46:50.189645 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:46:50.189645 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:46:50.189645 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:46:50.189645 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:46:50.189645 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:46:50.189645 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:46:50.189645 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:46:50.189645 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:46:50.190455 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 10 11:46:50.190455 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 10 11:46:50.190455 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 10 11:46:50.190455 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 10 11:46:50.190455 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 10 11:46:50.190455 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 10 11:46:50.190455 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 10 11:46:50.190455 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Oct 10 11:46:50.192519 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 282/535] 10.222.0.46 () {56 vars in 1111 bytes} [Mon Oct 10 11:46:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 504 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:46:50.200153 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c30abf7-b909-4094-a093-9257c46906c3 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:50.211158 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:50.218560 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0ca984-f5f0-4757-91e1-4d3827e64564 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:50.219651 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0ca984-f5f0-4757-91e1-4d3827e64564 None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:50.220157 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0ca984-f5f0-4757-91e1-4d3827e64564 None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:50.220658 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0ca984-f5f0-4757-91e1-4d3827e64564 None admin] PATH_INFO: `/v3/domains` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:50.221887 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0ca984-f5f0-4757-91e1-4d3827e64564 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:46:50.222444 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0ca984-f5f0-4757-91e1-4d3827e64564 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:50.230367 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0ca984-f5f0-4757-91e1-4d3827e64564 None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:50.268961 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 254/536] 10.222.0.46 () {56 vars in 1109 bytes} [Mon Oct 10 11:46:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 432 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:46:50.274992 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0ca984-f5f0-4757-91e1-4d3827e64564 None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:50.319155 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:50.525647 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c30abf7-b909-4094-a093-9257c46906c3 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:50.531105 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0ca984-f5f0-4757-91e1-4d3827e64564 None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:50.572656 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76051ce-d673-4f0f-82bb-dce8becf511c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:50.573560 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76051ce-d673-4f0f-82bb-dce8becf511c None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:50.574978 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76051ce-d673-4f0f-82bb-dce8becf511c None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:50.575431 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76051ce-d673-4f0f-82bb-dce8becf511c None admin] PATH_INFO: `/v3/projects` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:50.576297 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76051ce-d673-4f0f-82bb-dce8becf511c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:46:50.577671 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76051ce-d673-4f0f-82bb-dce8becf511c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:50.579352 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76051ce-d673-4f0f-82bb-dce8becf511c None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:50.613125 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 283/537] 10.222.0.46 () {56 vars in 1148 bytes} [Mon Oct 10 11:46:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 413 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:46:50.619375 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ebdef9-f703-47db-8b35-bfb263b94244 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:50.621375 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76051ce-d673-4f0f-82bb-dce8becf511c None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:50.626549 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ebdef9-f703-47db-8b35-bfb263b94244 None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:50.635657 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ebdef9-f703-47db-8b35-bfb263b94244 None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:50.636000 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ebdef9-f703-47db-8b35-bfb263b94244 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:50.660515 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ebdef9-f703-47db-8b35-bfb263b94244 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:50.663311 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ebdef9-f703-47db-8b35-bfb263b94244 None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:50.694528 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:50.695283 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application [None req-31ebdef9-f703-47db-8b35-bfb263b94244 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 10 11:46:50.695283 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:46:50.695283 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:46:50.695283 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:46:50.695283 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:46:50.695283 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:46:50.695283 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:46:50.695283 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:46:50.695283 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:46:50.695283 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:46:50.695283 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:46:50.695283 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:46:50.695283 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 10 11:46:50.695283 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 10 11:46:50.695283 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 10 11:46:50.696101 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 10 11:46:50.696101 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:46:50.696101 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:46:50.696101 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:46:50.696101 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:46:50.696101 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:46:50.696101 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:46:50.696101 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:46:50.696101 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application with Lock( Oct 10 11:46:50.696101 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:46:50.696101 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:46:50.696101 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:46:50.696101 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:46:50.696101 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:46:50.696101 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:46:50.696101 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:46:50.696101 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:46:50.696905 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 10 11:46:50.696905 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 10 11:46:50.696905 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 10 11:46:50.696905 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 10 11:46:50.696905 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 10 11:46:50.696905 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 10 11:46:50.696905 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 10 11:46:50.696905 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Oct 10 11:46:50.697727 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 255/538] 10.222.0.46 () {56 vars in 1111 bytes} [Mon Oct 10 11:46:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 424 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:46:50.707053 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ebdef9-f703-47db-8b35-bfb263b94244 None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:50.747028 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76051ce-d673-4f0f-82bb-dce8becf511c None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:50.789122 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf3cd66-ffbc-4894-884b-cd44b9685f94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:50.790037 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf3cd66-ffbc-4894-884b-cd44b9685f94 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:50.790500 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf3cd66-ffbc-4894-884b-cd44b9685f94 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:50.791130 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf3cd66-ffbc-4894-884b-cd44b9685f94 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:50.792446 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf3cd66-ffbc-4894-884b-cd44b9685f94 None admin] RBAC: Adding query filter params (role.id=3c35be38787740f4834c728a2ce73b6b, scope.project.id=4ad3772f7b6a47c28b7ba37a9316a3c1, user.id=5f130bbb69e546ce99b4e14de81d7e54) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:46:50.792963 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf3cd66-ffbc-4894-884b-cd44b9685f94 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:50.794615 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf3cd66-ffbc-4894-884b-cd44b9685f94 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:50.802758 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 284/539] 10.222.0.46 () {56 vars in 1374 bytes} [Mon Oct 10 11:46:50 2022] GET /identity/v3/role_assignments?user.id=5f130bbb69e546ce99b4e14de81d7e54&scope.project.id=4ad3772f7b6a47c28b7ba37a9316a3c1&role.id=3c35be38787740f4834c728a2ce73b6b => generated 259 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 11:46:50.804865 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cb966b-a35d-4c29-b6aa-1830496ebee5 None None] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:50.805625 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cb966b-a35d-4c29-b6aa-1830496ebee5 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:50.806108 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cb966b-a35d-4c29-b6aa-1830496ebee5 None None] PATH_INFO: `/` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:50.808012 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 285/540] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 10 11:46:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:46:50.813663 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0a9fd9-86b6-46f8-90f1-6b936351efc1 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:50.814253 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0a9fd9-86b6-46f8-90f1-6b936351efc1 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:50.814740 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0a9fd9-86b6-46f8-90f1-6b936351efc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:50.831842 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:50.854353 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-cf0a9fd9-86b6-46f8-90f1-6b936351efc1 None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:46:50.855835 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-cf0a9fd9-86b6-46f8-90f1-6b936351efc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:46:50.862594 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ebdef9-f703-47db-8b35-bfb263b94244 None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:50.910313 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f10427-5f33-411d-a032-b415dfd72d70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:50.912177 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f10427-5f33-411d-a032-b415dfd72d70 None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:50.912842 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f10427-5f33-411d-a032-b415dfd72d70 None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:50.913351 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f10427-5f33-411d-a032-b415dfd72d70 None admin] PATH_INFO: `/v3/projects` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:50.914371 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f10427-5f33-411d-a032-b415dfd72d70 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:46:50.914965 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f10427-5f33-411d-a032-b415dfd72d70 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:50.917190 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f10427-5f33-411d-a032-b415dfd72d70 None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:50.947739 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 256/541] 10.222.0.46 () {56 vars in 1148 bytes} [Mon Oct 10 11:46:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 11:46:50.959794 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f10427-5f33-411d-a032-b415dfd72d70 None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:50.991777 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 286/542] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 10 11:46:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 10 11:46:51.098865 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f10427-5f33-411d-a032-b415dfd72d70 None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:51.126342 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33d02c0-34ab-44da-a349-94d0c9c42787 None None] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:51.126802 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33d02c0-34ab-44da-a349-94d0c9c42787 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:51.127221 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33d02c0-34ab-44da-a349-94d0c9c42787 None None] PATH_INFO: `/` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:51.134858 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 287/543] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 10 11:46:51 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 10 11:46:51.141164 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e693f60c-0e09-48af-b22e-a7f0e811a00b None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:51.141599 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e693f60c-0e09-48af-b22e-a7f0e811a00b None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:51.142028 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e693f60c-0e09-48af-b22e-a7f0e811a00b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:51.156174 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f367a45-e2f9-48aa-a0f0-a1c9caeaef6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:51.157272 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f367a45-e2f9-48aa-a0f0-a1c9caeaef6b None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:51.157764 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f367a45-e2f9-48aa-a0f0-a1c9caeaef6b None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:51.158336 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f367a45-e2f9-48aa-a0f0-a1c9caeaef6b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:51.159392 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f367a45-e2f9-48aa-a0f0-a1c9caeaef6b None admin] RBAC: Adding query filter params (role.id=3c35be38787740f4834c728a2ce73b6b, scope.project.id=4ad3772f7b6a47c28b7ba37a9316a3c1, user.id=124f4aa8ab244ca2aadf4f2614df51e7) {{(pid=81676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:46:51.159962 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f367a45-e2f9-48aa-a0f0-a1c9caeaef6b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:51.161730 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f367a45-e2f9-48aa-a0f0-a1c9caeaef6b None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:51.169040 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 257/544] 10.222.0.46 () {56 vars in 1374 bytes} [Mon Oct 10 11:46:50 2022] GET /identity/v3/role_assignments?user.id=124f4aa8ab244ca2aadf4f2614df51e7&scope.project.id=4ad3772f7b6a47c28b7ba37a9316a3c1&role.id=3c35be38787740f4834c728a2ce73b6b => generated 259 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:46:51.196360 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-e693f60c-0e09-48af-b22e-a7f0e811a00b None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:46:51.198137 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-e693f60c-0e09-48af-b22e-a7f0e811a00b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:46:51.287498 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:51.331388 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 288/545] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 10 11:46:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 10 11:46:51.413665 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f367a45-e2f9-48aa-a0f0-a1c9caeaef6b None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:51.419116 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:51.648086 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f367a45-e2f9-48aa-a0f0-a1c9caeaef6b None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:51.724393 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737010e6-501f-4f7f-83ce-eff662fdfa87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:51.725386 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737010e6-501f-4f7f-83ce-eff662fdfa87 None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:51.725898 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737010e6-501f-4f7f-83ce-eff662fdfa87 None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:51.726398 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737010e6-501f-4f7f-83ce-eff662fdfa87 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:51.747612 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737010e6-501f-4f7f-83ce-eff662fdfa87 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 10 11:46:51.753131 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application [None req-737010e6-501f-4f7f-83ce-eff662fdfa87 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 10 11:46:51.753131 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:46:51.753131 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:46:51.753131 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:46:51.753131 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:46:51.753131 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:46:51.753131 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:46:51.753131 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:46:51.753131 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:46:51.753131 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:46:51.753131 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:46:51.753131 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:46:51.753131 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 10 11:46:51.753131 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 10 11:46:51.753131 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 10 11:46:51.753131 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 10 11:46:51.754065 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 10 11:46:51.754065 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 10 11:46:51.754065 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 10 11:46:51.754065 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 10 11:46:51.754065 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:46:51.754065 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:46:51.754065 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:46:51.754065 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:46:51.754065 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:46:51.754065 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:46:51.754065 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:46:51.754065 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application with Lock( Oct 10 11:46:51.754065 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:46:51.754065 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:46:51.754065 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:46:51.754065 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:46:51.754923 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:46:51.754923 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:46:51.754923 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:46:51.754923 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:46:51.754923 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 10 11:46:51.754923 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 10 11:46:51.754923 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 10 11:46:51.754923 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 10 11:46:51.754923 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 10 11:46:51.754923 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 10 11:46:51.754923 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 10 11:46:51.754923 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Oct 10 11:46:51.754923 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 258/546] 10.222.0.46 () {56 vars in 1102 bytes} [Mon Oct 10 11:46:51 2022] GET /identity/v3/roles/service => generated 85 bytes in 342 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:46:51.761379 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e693f60c-0e09-48af-b22e-a7f0e811a00b None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:51.765686 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737010e6-501f-4f7f-83ce-eff662fdfa87 None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:51.904433 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:51.941185 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e693f60c-0e09-48af-b22e-a7f0e811a00b None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:51.983447 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737010e6-501f-4f7f-83ce-eff662fdfa87 None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:52.007926 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13e3b13-927f-434c-ab1a-ed51056fd9cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:52.008928 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13e3b13-927f-434c-ab1a-ed51056fd9cf None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:52.009382 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13e3b13-927f-434c-ab1a-ed51056fd9cf None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:52.009832 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13e3b13-927f-434c-ab1a-ed51056fd9cf None admin] PATH_INFO: `/v3/roles` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:52.011028 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13e3b13-927f-434c-ab1a-ed51056fd9cf None admin] RBAC: Adding query filter params (name=service) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:46:52.011543 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13e3b13-927f-434c-ab1a-ed51056fd9cf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:52.014690 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13e3b13-927f-434c-ab1a-ed51056fd9cf None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:52.039756 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 289/547] 10.222.0.46 () {56 vars in 1103 bytes} [Mon Oct 10 11:46:51 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:46:52.045538 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13e3b13-927f-434c-ab1a-ed51056fd9cf None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:52.071426 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946b6360-4384-4d1f-83f2-d0df4e11afaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:52.072476 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946b6360-4384-4d1f-83f2-d0df4e11afaf None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:52.072940 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946b6360-4384-4d1f-83f2-d0df4e11afaf None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:52.073574 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946b6360-4384-4d1f-83f2-d0df4e11afaf None admin] PATH_INFO: `/v3/roles/service` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:52.102200 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:52.122685 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946b6360-4384-4d1f-83f2-d0df4e11afaf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 10 11:46:52.135243 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application [None req-946b6360-4384-4d1f-83f2-d0df4e11afaf None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 10 11:46:52.135243 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:46:52.135243 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:46:52.135243 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:46:52.135243 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:46:52.135243 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:46:52.135243 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:46:52.135243 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:46:52.135243 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:46:52.135243 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:46:52.135243 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:46:52.135243 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:46:52.135243 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 10 11:46:52.135243 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 10 11:46:52.135243 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 10 11:46:52.135243 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 10 11:46:52.136197 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 10 11:46:52.136197 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 10 11:46:52.136197 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 10 11:46:52.136197 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 10 11:46:52.136197 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:46:52.136197 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:46:52.136197 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:46:52.136197 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:46:52.136197 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:46:52.136197 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:46:52.136197 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:46:52.136197 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application with Lock( Oct 10 11:46:52.136197 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:46:52.136197 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:46:52.136197 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:46:52.136197 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:46:52.137475 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:46:52.137475 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:46:52.137475 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:46:52.137475 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:46:52.137475 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 10 11:46:52.137475 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 10 11:46:52.137475 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 10 11:46:52.137475 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 10 11:46:52.137475 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 10 11:46:52.137475 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 10 11:46:52.137475 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 10 11:46:52.137475 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Oct 10 11:46:52.138255 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 259/548] 10.222.0.46 () {56 vars in 1102 bytes} [Mon Oct 10 11:46:51 2022] GET /identity/v3/roles/service => generated 85 bytes in 372 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:46:52.146710 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946b6360-4384-4d1f-83f2-d0df4e11afaf None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:52.258024 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13e3b13-927f-434c-ab1a-ed51056fd9cf None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:52.319929 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b933579-d9e2-402d-a552-26c5772db177 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:52.320573 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b933579-d9e2-402d-a552-26c5772db177 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:52.320858 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b933579-d9e2-402d-a552-26c5772db177 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:52.321136 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b933579-d9e2-402d-a552-26c5772db177 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:52.335936 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b933579-d9e2-402d-a552-26c5772db177 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:52.337547 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b933579-d9e2-402d-a552-26c5772db177 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:52.354511 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application [None req-6b933579-d9e2-402d-a552-26c5772db177 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:46:52.354511 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:46:52.354511 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:46:52.354511 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:46:52.354511 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:46:52.354511 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:46:52.354511 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:46:52.354511 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:46:52.354511 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:46:52.354511 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:46:52.354511 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:46:52.354511 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:46:52.354511 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 10 11:46:52.354511 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 10 11:46:52.354511 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 10 11:46:52.354511 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 10 11:46:52.355497 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:46:52.355497 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:46:52.355497 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:46:52.355497 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:46:52.355497 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:46:52.355497 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:46:52.355497 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:46:52.355497 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application with Lock( Oct 10 11:46:52.355497 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:46:52.355497 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:46:52.355497 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:46:52.355497 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:46:52.355497 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:46:52.355497 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:46:52.355497 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:46:52.355497 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:46:52.355497 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 10 11:46:52.356335 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 10 11:46:52.356335 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:46:52.356335 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Oct 10 11:46:52.356335 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 290/549] 10.222.0.46 () {56 vars in 1108 bytes} [Mon Oct 10 11:46:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 311 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:46:52.364544 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b933579-d9e2-402d-a552-26c5772db177 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:52.403662 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946b6360-4384-4d1f-83f2-d0df4e11afaf None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:52.442523 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:52.475158 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc11f59-3ed8-4648-95e8-40f9aa26d98e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:52.476219 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc11f59-3ed8-4648-95e8-40f9aa26d98e None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:52.476719 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc11f59-3ed8-4648-95e8-40f9aa26d98e None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:52.479982 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc11f59-3ed8-4648-95e8-40f9aa26d98e None admin] PATH_INFO: `/v3/roles` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:52.479982 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc11f59-3ed8-4648-95e8-40f9aa26d98e None admin] RBAC: Adding query filter params (name=service) {{(pid=81676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:46:52.479982 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc11f59-3ed8-4648-95e8-40f9aa26d98e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:52.482222 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc11f59-3ed8-4648-95e8-40f9aa26d98e None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:52.561273 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 260/550] 10.222.0.46 () {56 vars in 1103 bytes} [Mon Oct 10 11:46:52 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 416 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:46:52.567668 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc11f59-3ed8-4648-95e8-40f9aa26d98e None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:52.622753 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:52.628629 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b933579-d9e2-402d-a552-26c5772db177 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:52.671017 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca4ed11-8410-4f65-b98d-56b263ce2c7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:52.671651 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca4ed11-8410-4f65-b98d-56b263ce2c7a None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:52.671941 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca4ed11-8410-4f65-b98d-56b263ce2c7a None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:52.672231 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca4ed11-8410-4f65-b98d-56b263ce2c7a None admin] PATH_INFO: `/v3/domains` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:52.672907 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca4ed11-8410-4f65-b98d-56b263ce2c7a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:46:52.673216 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca4ed11-8410-4f65-b98d-56b263ce2c7a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:52.674627 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca4ed11-8410-4f65-b98d-56b263ce2c7a None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:52.689082 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 291/551] 10.222.0.46 () {56 vars in 1109 bytes} [Mon Oct 10 11:46:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 325 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:46:52.690285 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d93dbf-57d8-40c9-9a23-ab76562068ca None None] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:52.690623 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d93dbf-57d8-40c9-9a23-ab76562068ca None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:52.690888 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d93dbf-57d8-40c9-9a23-ab76562068ca None None] PATH_INFO: `/` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:52.692888 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 292/552] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 10 11:46:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:46:52.697210 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d93dbf-57d8-40c9-9a23-ab76562068ca None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:52.771238 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc11f59-3ed8-4648-95e8-40f9aa26d98e None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:52.809926 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78df25ee-d537-429f-8c10-4c60d0f56fd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:52.810875 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78df25ee-d537-429f-8c10-4c60d0f56fd0 None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:52.811176 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78df25ee-d537-429f-8c10-4c60d0f56fd0 None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:52.811511 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78df25ee-d537-429f-8c10-4c60d0f56fd0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:52.825961 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78df25ee-d537-429f-8c10-4c60d0f56fd0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:52.828006 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78df25ee-d537-429f-8c10-4c60d0f56fd0 None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:52.850207 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application [None req-78df25ee-d537-429f-8c10-4c60d0f56fd0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:46:52.850207 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:46:52.850207 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:46:52.850207 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:46:52.850207 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:46:52.850207 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:46:52.850207 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:46:52.850207 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:46:52.850207 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:46:52.850207 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:46:52.850207 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:46:52.850207 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:46:52.850207 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 10 11:46:52.850207 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 10 11:46:52.850207 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 10 11:46:52.850207 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 10 11:46:52.851193 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:46:52.851193 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:46:52.851193 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:46:52.851193 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:46:52.851193 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:46:52.851193 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:46:52.851193 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:46:52.851193 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application with Lock( Oct 10 11:46:52.851193 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:46:52.851193 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:46:52.851193 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:46:52.851193 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:46:52.851193 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:46:52.851193 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:46:52.851193 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:46:52.851193 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:46:52.851193 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 10 11:46:52.852107 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 10 11:46:52.852107 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:46:52.852107 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Oct 10 11:46:52.852107 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 261/553] 10.222.0.46 () {56 vars in 1108 bytes} [Mon Oct 10 11:46:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:46:52.853656 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075061a6-3f21-4b03-8b90-53ecbcb9e90e None None] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:52.853948 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075061a6-3f21-4b03-8b90-53ecbcb9e90e None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:52.857154 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075061a6-3f21-4b03-8b90-53ecbcb9e90e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:52.877746 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.auth.core [None req-075061a6-3f21-4b03-8b90-53ecbcb9e90e None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:46:52.879133 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-075061a6-3f21-4b03-8b90-53ecbcb9e90e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:46:52.898806 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d93dbf-57d8-40c9-9a23-ab76562068ca None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:52.915083 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:52.960250 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 262/554] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 10 11:46:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 10 11:46:52.961373 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075061a6-3f21-4b03-8b90-53ecbcb9e90e None None] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:52.972368 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd5ef36-1e6f-4bee-83d3-2dc3087c87ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:52.973445 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd5ef36-1e6f-4bee-83d3-2dc3087c87ae None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:52.973913 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd5ef36-1e6f-4bee-83d3-2dc3087c87ae None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:52.974375 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd5ef36-1e6f-4bee-83d3-2dc3087c87ae None admin] PATH_INFO: `/v3/users/swift` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:52.995147 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd5ef36-1e6f-4bee-83d3-2dc3087c87ae None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:52.998211 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd5ef36-1e6f-4bee-83d3-2dc3087c87ae None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:53.043694 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application [None req-6bd5ef36-1e6f-4bee-83d3-2dc3087c87ae None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 10 11:46:53.043694 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:46:53.043694 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:46:53.043694 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:46:53.043694 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:46:53.043694 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:46:53.043694 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:46:53.043694 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:46:53.043694 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:46:53.043694 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:46:53.043694 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:46:53.043694 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:46:53.043694 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 10 11:46:53.043694 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 10 11:46:53.043694 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 10 11:46:53.043694 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 10 11:46:53.044886 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:46:53.044886 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:46:53.044886 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 10 11:46:53.044886 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 10 11:46:53.044886 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 10 11:46:53.044886 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 10 11:46:53.044886 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:46:53.044886 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:46:53.044886 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:46:53.044886 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:46:53.044886 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:46:53.044886 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application with Lock( Oct 10 11:46:53.044886 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:46:53.044886 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:46:53.044886 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:46:53.044886 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:46:53.044886 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:46:53.044886 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:46:53.045974 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:46:53.045974 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:46:53.045974 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 10 11:46:53.045974 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 10 11:46:53.045974 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 10 11:46:53.045974 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 10 11:46:53.045974 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 10 11:46:53.045974 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 10 11:46:53.045974 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 10 11:46:53.045974 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Oct 10 11:46:53.046522 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 293/555] 10.222.0.46 () {56 vars in 1096 bytes} [Mon Oct 10 11:46:52 2022] GET /identity/v3/users/swift => generated 83 bytes in 350 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 10 11:46:53.055242 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd5ef36-1e6f-4bee-83d3-2dc3087c87ae None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:53.078246 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075061a6-3f21-4b03-8b90-53ecbcb9e90e None None] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:53.120328 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:53.128429 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8262f0eb-e1b7-443b-a91f-5edd652803ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:53.129382 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8262f0eb-e1b7-443b-a91f-5edd652803ea None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:53.129836 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8262f0eb-e1b7-443b-a91f-5edd652803ea None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:53.130244 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8262f0eb-e1b7-443b-a91f-5edd652803ea None admin] PATH_INFO: `/v3/domains` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:53.131094 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8262f0eb-e1b7-443b-a91f-5edd652803ea None admin] RBAC: Adding query filter params (name=Default) {{(pid=81676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:46:53.131548 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8262f0eb-e1b7-443b-a91f-5edd652803ea None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:53.133113 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8262f0eb-e1b7-443b-a91f-5edd652803ea None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:53.172205 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 263/556] 10.222.0.46 () {56 vars in 1109 bytes} [Mon Oct 10 11:46:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:46:53.178610 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8262f0eb-e1b7-443b-a91f-5edd652803ea None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:53.246076 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:53.252052 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd5ef36-1e6f-4bee-83d3-2dc3087c87ae None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:53.290343 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c420cd-b0b9-462c-8cb0-dd76fa0ee6d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:53.291229 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c420cd-b0b9-462c-8cb0-dd76fa0ee6d1 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:53.291764 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c420cd-b0b9-462c-8cb0-dd76fa0ee6d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:53.292045 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c420cd-b0b9-462c-8cb0-dd76fa0ee6d1 None admin] PATH_INFO: `/v3/users` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:53.292865 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c420cd-b0b9-462c-8cb0-dd76fa0ee6d1 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:46:53.293255 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c420cd-b0b9-462c-8cb0-dd76fa0ee6d1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:53.295255 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c420cd-b0b9-462c-8cb0-dd76fa0ee6d1 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:53.319658 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 294/557] 10.222.0.46 () {56 vars in 1135 bytes} [Mon Oct 10 11:46:53 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 11:46:53.326200 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c420cd-b0b9-462c-8cb0-dd76fa0ee6d1 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:53.426595 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8262f0eb-e1b7-443b-a91f-5edd652803ea None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:53.443140 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:53.481508 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c420cd-b0b9-462c-8cb0-dd76fa0ee6d1 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:53.505418 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818201f4-8cd6-4311-a131-16af91afac5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:53.506524 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818201f4-8cd6-4311-a131-16af91afac5f None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:53.507070 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818201f4-8cd6-4311-a131-16af91afac5f None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:53.508231 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818201f4-8cd6-4311-a131-16af91afac5f None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:53.527118 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818201f4-8cd6-4311-a131-16af91afac5f None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:53.531152 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818201f4-8cd6-4311-a131-16af91afac5f None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:53.536697 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a51dbb-e1cb-46fa-9c55-50407a50fb7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:53.537686 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a51dbb-e1cb-46fa-9c55-50407a50fb7c None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:53.538288 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a51dbb-e1cb-46fa-9c55-50407a50fb7c None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:53.538530 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a51dbb-e1cb-46fa-9c55-50407a50fb7c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:53.560808 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application [None req-818201f4-8cd6-4311-a131-16af91afac5f None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Oct 10 11:46:53.560808 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:46:53.560808 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:46:53.560808 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:46:53.560808 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:46:53.560808 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:46:53.560808 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:46:53.560808 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:46:53.560808 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:46:53.560808 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:46:53.560808 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:46:53.560808 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:46:53.560808 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 10 11:46:53.560808 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 10 11:46:53.560808 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 10 11:46:53.560808 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 10 11:46:53.562506 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:46:53.562506 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:46:53.562506 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 10 11:46:53.562506 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 10 11:46:53.562506 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 10 11:46:53.562506 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 10 11:46:53.562506 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:46:53.562506 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:46:53.562506 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:46:53.562506 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:46:53.562506 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:46:53.562506 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application with Lock( Oct 10 11:46:53.562506 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:46:53.562506 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:46:53.562506 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:46:53.562506 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:46:53.562506 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:46:53.562506 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:46:53.564116 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:46:53.564116 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:46:53.564116 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 10 11:46:53.564116 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 10 11:46:53.564116 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 10 11:46:53.564116 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 10 11:46:53.564116 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 10 11:46:53.564116 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 10 11:46:53.564116 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Oct 10 11:46:53.564116 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Oct 10 11:46:53.565171 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 264/558] 10.222.0.46 () {56 vars in 1099 bytes} [Mon Oct 10 11:46:53 2022] GET /identity/v3/users/cinder => generated 84 bytes in 387 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:46:53.566504 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818201f4-8cd6-4311-a131-16af91afac5f None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:53.663756 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a51dbb-e1cb-46fa-9c55-50407a50fb7c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:53.666539 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a51dbb-e1cb-46fa-9c55-50407a50fb7c None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:53.743824 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:53.773781 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818201f4-8cd6-4311-a131-16af91afac5f None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:53.788744 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:53.820520 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399e2a77-7c8b-43f4-8b2e-96b0545a302a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:53.821265 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399e2a77-7c8b-43f4-8b2e-96b0545a302a None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:53.821548 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399e2a77-7c8b-43f4-8b2e-96b0545a302a None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:53.821819 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399e2a77-7c8b-43f4-8b2e-96b0545a302a None admin] PATH_INFO: `/v3/roles/service` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:53.829108 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 295/559] 10.222.0.46 () {58 vars in 1282 bytes} [Mon Oct 10 11:46:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 503 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 10 11:46:53.830537 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a51dbb-e1cb-46fa-9c55-50407a50fb7c None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:53.839360 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399e2a77-7c8b-43f4-8b2e-96b0545a302a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 10 11:46:53.844395 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application [None req-399e2a77-7c8b-43f4-8b2e-96b0545a302a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 10 11:46:53.844395 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:46:53.844395 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:46:53.844395 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:46:53.844395 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:46:53.844395 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:46:53.844395 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:46:53.844395 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:46:53.844395 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:46:53.844395 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:46:53.844395 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:46:53.844395 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:46:53.844395 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 10 11:46:53.844395 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 10 11:46:53.844395 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 10 11:46:53.844395 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 10 11:46:53.845392 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 10 11:46:53.845392 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 10 11:46:53.845392 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 10 11:46:53.845392 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 10 11:46:53.845392 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:46:53.845392 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:46:53.845392 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:46:53.845392 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:46:53.845392 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:46:53.845392 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:46:53.845392 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:46:53.845392 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application with Lock( Oct 10 11:46:53.845392 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:46:53.845392 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:46:53.845392 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:46:53.845392 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:46:53.846198 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:46:53.846198 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:46:53.846198 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:46:53.846198 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:46:53.846198 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 10 11:46:53.846198 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 10 11:46:53.846198 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 10 11:46:53.846198 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 10 11:46:53.846198 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 10 11:46:53.846198 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 10 11:46:53.846198 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 10 11:46:53.846198 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Oct 10 11:46:53.846198 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 265/560] 10.222.0.46 () {56 vars in 1102 bytes} [Mon Oct 10 11:46:53 2022] GET /identity/v3/roles/service => generated 85 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:46:53.846903 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b75075-6a33-440c-8494-66103c7b79a6 None None] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:53.847531 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b75075-6a33-440c-8494-66103c7b79a6 None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:53.847531 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b75075-6a33-440c-8494-66103c7b79a6 None None] PATH_INFO: `/` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:53.848765 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 266/561] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 10 11:46:53 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:46:53.850588 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b75075-6a33-440c-8494-66103c7b79a6 None None] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:54.000771 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a51dbb-e1cb-46fa-9c55-50407a50fb7c None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:54.016277 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b75075-6a33-440c-8494-66103c7b79a6 None None] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:54.057489 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b5608f-72c4-4469-9191-4b1e84d93e0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:54.058340 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b5608f-72c4-4469-9191-4b1e84d93e0f None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:54.058621 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b5608f-72c4-4469-9191-4b1e84d93e0f None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:54.058956 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b5608f-72c4-4469-9191-4b1e84d93e0f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:54.068077 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c25373-cad9-4ff1-858e-ad9d7ad2bd6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:54.069158 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c25373-cad9-4ff1-858e-ad9d7ad2bd6d None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:54.069627 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c25373-cad9-4ff1-858e-ad9d7ad2bd6d None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:54.070116 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c25373-cad9-4ff1-858e-ad9d7ad2bd6d None admin] PATH_INFO: `/v3/users` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:54.071608 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c25373-cad9-4ff1-858e-ad9d7ad2bd6d None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:46:54.071954 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c25373-cad9-4ff1-858e-ad9d7ad2bd6d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:54.073295 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b5608f-72c4-4469-9191-4b1e84d93e0f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:54.074815 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b5608f-72c4-4469-9191-4b1e84d93e0f None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:54.074991 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c25373-cad9-4ff1-858e-ad9d7ad2bd6d None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:54.083866 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:54.098728 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application [None req-61b5608f-72c4-4469-9191-4b1e84d93e0f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:46:54.098728 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:46:54.098728 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:46:54.098728 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:46:54.098728 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:46:54.098728 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:46:54.098728 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:46:54.098728 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:46:54.098728 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:46:54.098728 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:46:54.098728 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:46:54.098728 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:46:54.098728 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 10 11:46:54.098728 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 10 11:46:54.098728 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 10 11:46:54.098728 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 10 11:46:54.107293 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:46:54.107293 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:46:54.107293 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:46:54.107293 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:46:54.107293 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:46:54.107293 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:46:54.107293 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:46:54.107293 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application with Lock( Oct 10 11:46:54.107293 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:46:54.107293 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:46:54.107293 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:46:54.107293 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:46:54.107293 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:46:54.107293 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:46:54.107293 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:46:54.107293 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:46:54.107293 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 10 11:46:54.108860 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 10 11:46:54.108860 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:46:54.108860 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Oct 10 11:46:54.108860 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 267/562] 10.222.0.46 () {56 vars in 1108 bytes} [Mon Oct 10 11:46:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:46:54.108860 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66405dd-3926-4160-86c2-fa913817e325 None None] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:54.108860 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66405dd-3926-4160-86c2-fa913817e325 None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:54.108860 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66405dd-3926-4160-86c2-fa913817e325 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:54.122263 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 296/563] 10.222.0.46 () {56 vars in 1137 bytes} [Mon Oct 10 11:46:53 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:46:54.122263 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c25373-cad9-4ff1-858e-ad9d7ad2bd6d None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:54.140778 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.auth.core [None req-b66405dd-3926-4160-86c2-fa913817e325 None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:46:54.142843 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-b66405dd-3926-4160-86c2-fa913817e325 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:46:54.171853 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:54.235840 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 268/564] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 10 11:46:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 10 11:46:54.236708 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66405dd-3926-4160-86c2-fa913817e325 None None] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:54.285961 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c25373-cad9-4ff1-858e-ad9d7ad2bd6d None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:54.347565 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8044551-7aac-43fc-a478-7f3916628461 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:54.348565 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8044551-7aac-43fc-a478-7f3916628461 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:54.349039 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8044551-7aac-43fc-a478-7f3916628461 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:54.349504 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8044551-7aac-43fc-a478-7f3916628461 None admin] PATH_INFO: `/v3/roles` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:54.350742 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8044551-7aac-43fc-a478-7f3916628461 None admin] RBAC: Adding query filter params (name=service) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:46:54.351268 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8044551-7aac-43fc-a478-7f3916628461 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:54.353753 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8044551-7aac-43fc-a478-7f3916628461 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:54.369517 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 297/565] 10.222.0.46 () {56 vars in 1103 bytes} [Mon Oct 10 11:46:54 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:46:54.371274 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8044551-7aac-43fc-a478-7f3916628461 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:54.381642 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66405dd-3926-4160-86c2-fa913817e325 None None] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:54.423302 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:54.436837 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f49a6d5-1a2a-4775-9daf-f9e30bc21c1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:54.437932 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f49a6d5-1a2a-4775-9daf-f9e30bc21c1a None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:54.438509 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f49a6d5-1a2a-4775-9daf-f9e30bc21c1a None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:54.439356 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f49a6d5-1a2a-4775-9daf-f9e30bc21c1a None admin] PATH_INFO: `/v3/domains` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:54.440238 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f49a6d5-1a2a-4775-9daf-f9e30bc21c1a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:46:54.440770 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f49a6d5-1a2a-4775-9daf-f9e30bc21c1a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:54.443388 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f49a6d5-1a2a-4775-9daf-f9e30bc21c1a None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:54.477440 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 269/566] 10.222.0.46 () {56 vars in 1109 bytes} [Mon Oct 10 11:46:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:46:54.477440 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79c62ba-8743-426d-aa80-804bc2d17603 None None] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:54.477440 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79c62ba-8743-426d-aa80-804bc2d17603 None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:54.477440 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79c62ba-8743-426d-aa80-804bc2d17603 None None] PATH_INFO: `/` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:54.478026 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 270/567] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 10 11:46:54 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:46:54.479395 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79c62ba-8743-426d-aa80-804bc2d17603 None None] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:54.536272 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:54.542869 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8044551-7aac-43fc-a478-7f3916628461 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:54.604728 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9856617-1ce5-4db1-bc84-c6984144da7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:54.605507 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9856617-1ce5-4db1-bc84-c6984144da7e None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:54.605850 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9856617-1ce5-4db1-bc84-c6984144da7e None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:54.606241 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9856617-1ce5-4db1-bc84-c6984144da7e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:54.610703 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79c62ba-8743-426d-aa80-804bc2d17603 None None] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:54.619991 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9856617-1ce5-4db1-bc84-c6984144da7e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:54.621963 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9856617-1ce5-4db1-bc84-c6984144da7e None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:54.639861 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application [None req-e9856617-1ce5-4db1-bc84-c6984144da7e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:46:54.639861 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:46:54.639861 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:46:54.639861 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:46:54.639861 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:46:54.639861 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:46:54.639861 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:46:54.639861 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:46:54.639861 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:46:54.639861 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:46:54.639861 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:46:54.639861 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:46:54.639861 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 10 11:46:54.639861 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 10 11:46:54.639861 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 10 11:46:54.639861 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 10 11:46:54.641765 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:46:54.641765 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:46:54.641765 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:46:54.641765 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:46:54.641765 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:46:54.641765 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:46:54.641765 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:46:54.641765 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application with Lock( Oct 10 11:46:54.641765 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:46:54.641765 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:46:54.641765 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:46:54.641765 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:46:54.641765 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:46:54.641765 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:46:54.641765 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:46:54.641765 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:46:54.641765 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 10 11:46:54.644411 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 10 11:46:54.644411 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:46:54.644411 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Oct 10 11:46:54.644411 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 298/568] 10.222.0.46 () {56 vars in 1108 bytes} [Mon Oct 10 11:46:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:46:54.644411 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9856617-1ce5-4db1-bc84-c6984144da7e None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:54.653348 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cf2b5a-7bda-43f8-8767-a05cda41ae2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:54.654002 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cf2b5a-7bda-43f8-8767-a05cda41ae2a None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:54.654274 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cf2b5a-7bda-43f8-8767-a05cda41ae2a None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:54.655096 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cf2b5a-7bda-43f8-8767-a05cda41ae2a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:54.667261 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69cf2b5a-7bda-43f8-8767-a05cda41ae2a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:54.669155 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69cf2b5a-7bda-43f8-8767-a05cda41ae2a None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:54.688943 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application [None req-69cf2b5a-7bda-43f8-8767-a05cda41ae2a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:46:54.688943 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:46:54.688943 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:46:54.688943 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:46:54.688943 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:46:54.688943 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:46:54.688943 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:46:54.688943 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:46:54.688943 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:46:54.688943 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:46:54.688943 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:46:54.688943 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:46:54.688943 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 10 11:46:54.688943 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 10 11:46:54.688943 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 10 11:46:54.688943 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 10 11:46:54.690852 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:46:54.690852 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:46:54.690852 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:46:54.690852 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:46:54.690852 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:46:54.690852 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:46:54.690852 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:46:54.690852 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application with Lock( Oct 10 11:46:54.690852 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:46:54.690852 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:46:54.690852 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:46:54.690852 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:46:54.690852 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:46:54.690852 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:46:54.690852 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:46:54.690852 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:46:54.690852 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 10 11:46:54.693567 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 10 11:46:54.693567 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:46:54.693567 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Oct 10 11:46:54.693567 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 271/569] 10.222.0.46 () {56 vars in 1108 bytes} [Mon Oct 10 11:46:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:46:54.693567 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a7b39c-b7d9-47ce-8a4e-56257876d73b None None] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:54.694226 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a7b39c-b7d9-47ce-8a4e-56257876d73b None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:54.694226 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a7b39c-b7d9-47ce-8a4e-56257876d73b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:54.716575 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.auth.core [None req-b7a7b39c-b7d9-47ce-8a4e-56257876d73b None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:46:54.718439 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-b7a7b39c-b7d9-47ce-8a4e-56257876d73b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:46:54.773594 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:54.802904 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9856617-1ce5-4db1-bc84-c6984144da7e None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:54.838783 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 272/570] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 10 11:46:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 10 11:46:54.838783 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a7b39c-b7d9-47ce-8a4e-56257876d73b None None] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:54.871859 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd5128d-1f21-4b94-b7b1-eb6199d1f976 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:54.873105 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd5128d-1f21-4b94-b7b1-eb6199d1f976 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:54.873682 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd5128d-1f21-4b94-b7b1-eb6199d1f976 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:54.874138 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd5128d-1f21-4b94-b7b1-eb6199d1f976 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:54.887960 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd5128d-1f21-4b94-b7b1-eb6199d1f976 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:54.891212 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd5128d-1f21-4b94-b7b1-eb6199d1f976 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:54.906912 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application [None req-1bd5128d-1f21-4b94-b7b1-eb6199d1f976 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 10 11:46:54.906912 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:46:54.906912 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:46:54.906912 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:46:54.906912 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:46:54.906912 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:46:54.906912 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:46:54.906912 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:46:54.906912 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:46:54.906912 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:46:54.906912 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:46:54.906912 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:46:54.906912 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 10 11:46:54.906912 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 10 11:46:54.906912 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 10 11:46:54.908634 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 10 11:46:54.908634 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:46:54.908634 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:46:54.908634 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:46:54.908634 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:46:54.908634 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:46:54.908634 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:46:54.908634 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:46:54.908634 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application with Lock( Oct 10 11:46:54.908634 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:46:54.908634 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:46:54.908634 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:46:54.908634 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:46:54.908634 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:46:54.908634 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:46:54.908634 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:46:54.908634 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:46:54.911099 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 10 11:46:54.911099 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 10 11:46:54.911099 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 10 11:46:54.911099 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 10 11:46:54.911099 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 10 11:46:54.911099 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 10 11:46:54.911099 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 10 11:46:54.911099 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Oct 10 11:46:54.911099 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 299/571] 10.222.0.46 () {56 vars in 1111 bytes} [Mon Oct 10 11:46:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:46:54.913134 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd5128d-1f21-4b94-b7b1-eb6199d1f976 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:54.932638 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:55.002397 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a7b39c-b7d9-47ce-8a4e-56257876d73b None None] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:55.041095 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd5128d-1f21-4b94-b7b1-eb6199d1f976 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:55.068528 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67c5d08-b4bc-4250-8a17-2c275befe76a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:55.069489 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67c5d08-b4bc-4250-8a17-2c275befe76a None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:55.069985 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67c5d08-b4bc-4250-8a17-2c275befe76a None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:55.070451 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67c5d08-b4bc-4250-8a17-2c275befe76a None admin] PATH_INFO: `/v3/roles/service` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:55.085985 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec9b302-487c-4754-a51c-9cec599d1143 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:55.086267 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67c5d08-b4bc-4250-8a17-2c275befe76a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 10 11:46:55.087118 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec9b302-487c-4754-a51c-9cec599d1143 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:55.087623 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec9b302-487c-4754-a51c-9cec599d1143 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:55.088086 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec9b302-487c-4754-a51c-9cec599d1143 None admin] PATH_INFO: `/v3/domains` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:55.089325 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec9b302-487c-4754-a51c-9cec599d1143 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:46:55.089863 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec9b302-487c-4754-a51c-9cec599d1143 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:55.091506 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application [None req-e67c5d08-b4bc-4250-8a17-2c275befe76a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 10 11:46:55.091506 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:46:55.091506 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:46:55.091506 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:46:55.091506 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:46:55.091506 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:46:55.091506 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:46:55.091506 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:46:55.091506 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:46:55.091506 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:46:55.091506 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:46:55.091506 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:46:55.091506 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 10 11:46:55.091506 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 10 11:46:55.091506 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 10 11:46:55.091506 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 10 11:46:55.094054 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 10 11:46:55.094054 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 10 11:46:55.094054 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 10 11:46:55.094054 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 10 11:46:55.094054 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:46:55.094054 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:46:55.094054 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:46:55.094054 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:46:55.094054 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:46:55.094054 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:46:55.094054 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:46:55.094054 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application with Lock( Oct 10 11:46:55.094054 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:46:55.094054 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:46:55.094054 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:46:55.094054 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:46:55.096827 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:46:55.096827 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:46:55.096827 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:46:55.096827 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:46:55.096827 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 10 11:46:55.096827 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 10 11:46:55.096827 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 10 11:46:55.096827 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 10 11:46:55.096827 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 10 11:46:55.096827 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 10 11:46:55.096827 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 10 11:46:55.096827 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Oct 10 11:46:55.098079 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec9b302-487c-4754-a51c-9cec599d1143 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:55.098285 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 273/572] 10.222.0.46 () {56 vars in 1102 bytes} [Mon Oct 10 11:46:54 2022] GET /identity/v3/roles/service => generated 85 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:46:55.098285 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67c5d08-b4bc-4250-8a17-2c275befe76a None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:55.126738 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 300/573] 10.222.0.46 () {56 vars in 1109 bytes} [Mon Oct 10 11:46:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:46:55.127569 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec9b302-487c-4754-a51c-9cec599d1143 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:55.149058 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:55.229090 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:55.268120 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec9b302-487c-4754-a51c-9cec599d1143 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:55.283600 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67c5d08-b4bc-4250-8a17-2c275befe76a None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:55.306449 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a1c637-09d7-4fe7-a374-0d4d99894b29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:55.307119 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a1c637-09d7-4fe7-a374-0d4d99894b29 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:55.307403 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a1c637-09d7-4fe7-a374-0d4d99894b29 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:55.307682 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a1c637-09d7-4fe7-a374-0d4d99894b29 None admin] PATH_INFO: `/v3/projects` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:55.308400 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a1c637-09d7-4fe7-a374-0d4d99894b29 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:46:55.308813 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a1c637-09d7-4fe7-a374-0d4d99894b29 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:55.310307 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a1c637-09d7-4fe7-a374-0d4d99894b29 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:55.324413 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 301/574] 10.222.0.46 () {56 vars in 1148 bytes} [Mon Oct 10 11:46:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:46:55.325071 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a1c637-09d7-4fe7-a374-0d4d99894b29 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:55.346911 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd63b80a-02ee-4f13-9b69-889926fa061f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:55.347895 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd63b80a-02ee-4f13-9b69-889926fa061f None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:55.348356 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd63b80a-02ee-4f13-9b69-889926fa061f None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:55.348814 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd63b80a-02ee-4f13-9b69-889926fa061f None admin] PATH_INFO: `/v3/domains` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:55.349946 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd63b80a-02ee-4f13-9b69-889926fa061f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:46:55.350368 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd63b80a-02ee-4f13-9b69-889926fa061f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:55.352885 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd63b80a-02ee-4f13-9b69-889926fa061f None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:55.378472 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 274/575] 10.222.0.46 () {56 vars in 1109 bytes} [Mon Oct 10 11:46:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:46:55.378472 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd63b80a-02ee-4f13-9b69-889926fa061f None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:55.477282 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a1c637-09d7-4fe7-a374-0d4d99894b29 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:55.500838 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:55.513023 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd63b80a-02ee-4f13-9b69-889926fa061f None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:55.530189 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:55.538319 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24d7f12-5b6a-4542-9011-fcd9b57d83a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:55.539861 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24d7f12-5b6a-4542-9011-fcd9b57d83a4 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:55.539861 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24d7f12-5b6a-4542-9011-fcd9b57d83a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:55.540133 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24d7f12-5b6a-4542-9011-fcd9b57d83a4 None admin] PATH_INFO: `/v3/roles` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:55.541178 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24d7f12-5b6a-4542-9011-fcd9b57d83a4 None admin] RBAC: Adding query filter params (name=service) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:46:55.541686 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24d7f12-5b6a-4542-9011-fcd9b57d83a4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:55.544013 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24d7f12-5b6a-4542-9011-fcd9b57d83a4 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:55.555316 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 302/576] 10.222.0.46 () {56 vars in 1103 bytes} [Mon Oct 10 11:46:55 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:46:55.556207 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24d7f12-5b6a-4542-9011-fcd9b57d83a4 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:55.591042 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b7fcb8-35c2-4cfa-8d43-80ab038c6053 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:55.592647 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b7fcb8-35c2-4cfa-8d43-80ab038c6053 None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:55.593501 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b7fcb8-35c2-4cfa-8d43-80ab038c6053 None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:55.594007 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b7fcb8-35c2-4cfa-8d43-80ab038c6053 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:55.606845 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b7fcb8-35c2-4cfa-8d43-80ab038c6053 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:55.610010 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b7fcb8-35c2-4cfa-8d43-80ab038c6053 None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:55.638647 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application [None req-c5b7fcb8-35c2-4cfa-8d43-80ab038c6053 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 10 11:46:55.638647 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:46:55.638647 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:46:55.638647 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:46:55.638647 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:46:55.638647 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:46:55.638647 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:46:55.638647 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:46:55.638647 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:46:55.638647 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:46:55.638647 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:46:55.638647 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:46:55.638647 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 10 11:46:55.638647 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 10 11:46:55.638647 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 10 11:46:55.640179 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 10 11:46:55.640179 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:46:55.640179 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:46:55.640179 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:46:55.640179 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:46:55.640179 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:46:55.640179 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:46:55.640179 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:46:55.640179 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application with Lock( Oct 10 11:46:55.640179 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:46:55.640179 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:46:55.640179 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:46:55.640179 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:46:55.640179 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:46:55.640179 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:46:55.640179 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:46:55.640179 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:46:55.641058 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 10 11:46:55.641058 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 10 11:46:55.641058 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 10 11:46:55.641058 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 10 11:46:55.641058 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 10 11:46:55.641058 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 10 11:46:55.641058 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 10 11:46:55.641058 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Oct 10 11:46:55.641955 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 275/577] 10.222.0.46 () {56 vars in 1111 bytes} [Mon Oct 10 11:46:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:46:55.643214 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b7fcb8-35c2-4cfa-8d43-80ab038c6053 None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:55.705989 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24d7f12-5b6a-4542-9011-fcd9b57d83a4 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:55.764934 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872fbbd0-648f-4dec-8664-4502a0b240d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:55.765814 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872fbbd0-648f-4dec-8664-4502a0b240d3 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:55.766187 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872fbbd0-648f-4dec-8664-4502a0b240d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:55.766527 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872fbbd0-648f-4dec-8664-4502a0b240d3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:55.778456 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872fbbd0-648f-4dec-8664-4502a0b240d3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 10 11:46:55.784601 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application [None req-872fbbd0-648f-4dec-8664-4502a0b240d3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 10 11:46:55.784601 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:46:55.784601 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:46:55.784601 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:46:55.784601 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:46:55.784601 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:46:55.784601 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:46:55.784601 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:46:55.784601 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:46:55.784601 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:46:55.784601 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:46:55.784601 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:46:55.784601 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 10 11:46:55.784601 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 10 11:46:55.784601 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 10 11:46:55.784601 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 10 11:46:55.791359 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 10 11:46:55.791359 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 10 11:46:55.791359 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 10 11:46:55.791359 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 10 11:46:55.791359 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:46:55.791359 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:46:55.791359 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:46:55.791359 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:46:55.791359 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:46:55.791359 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:46:55.791359 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:46:55.791359 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application with Lock( Oct 10 11:46:55.791359 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:46:55.791359 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:46:55.791359 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:46:55.791359 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:46:55.792326 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:46:55.792326 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:46:55.792326 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:46:55.792326 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:46:55.792326 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 10 11:46:55.792326 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 10 11:46:55.792326 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 10 11:46:55.792326 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 10 11:46:55.792326 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 10 11:46:55.792326 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 10 11:46:55.792326 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 10 11:46:55.792326 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Oct 10 11:46:55.792326 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 303/578] 10.222.0.46 () {56 vars in 1102 bytes} [Mon Oct 10 11:46:55 2022] GET /identity/v3/roles/service => generated 85 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:46:55.792326 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872fbbd0-648f-4dec-8664-4502a0b240d3 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:55.812187 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b7fcb8-35c2-4cfa-8d43-80ab038c6053 None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:55.867217 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:55.876601 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748a93ce-fe20-471d-baa6-37307d580d34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:55.877672 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748a93ce-fe20-471d-baa6-37307d580d34 None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:55.878164 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748a93ce-fe20-471d-baa6-37307d580d34 None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:55.878694 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748a93ce-fe20-471d-baa6-37307d580d34 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:55.880062 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748a93ce-fe20-471d-baa6-37307d580d34 None admin] RBAC: Adding query filter params (role.id=3c35be38787740f4834c728a2ce73b6b, scope.project.id=4ad3772f7b6a47c28b7ba37a9316a3c1, user.id=15ef3d53ea484237adb0b3d64fdc5480) {{(pid=81676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:46:55.880613 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748a93ce-fe20-471d-baa6-37307d580d34 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:55.883229 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748a93ce-fe20-471d-baa6-37307d580d34 None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:55.893746 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 276/579] 10.222.0.46 () {56 vars in 1374 bytes} [Mon Oct 10 11:46:55 2022] GET /identity/v3/role_assignments?user.id=15ef3d53ea484237adb0b3d64fdc5480&scope.project.id=4ad3772f7b6a47c28b7ba37a9316a3c1&role.id=3c35be38787740f4834c728a2ce73b6b => generated 259 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:46:55.895477 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748a93ce-fe20-471d-baa6-37307d580d34 None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:55.934027 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:55.956701 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872fbbd0-648f-4dec-8664-4502a0b240d3 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:56.006840 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b097cdd8-2da4-4c9f-9dd9-435782f10d81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:56.007866 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b097cdd8-2da4-4c9f-9dd9-435782f10d81 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:56.008290 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b097cdd8-2da4-4c9f-9dd9-435782f10d81 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:56.008751 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b097cdd8-2da4-4c9f-9dd9-435782f10d81 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:56.023077 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b097cdd8-2da4-4c9f-9dd9-435782f10d81 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:56.025567 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b097cdd8-2da4-4c9f-9dd9-435782f10d81 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:56.047921 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application [None req-b097cdd8-2da4-4c9f-9dd9-435782f10d81 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Oct 10 11:46:56.047921 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:46:56.047921 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:46:56.047921 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:46:56.047921 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:46:56.047921 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:46:56.047921 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:46:56.047921 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:46:56.047921 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:46:56.047921 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:46:56.047921 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:46:56.047921 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:46:56.047921 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 10 11:46:56.047921 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 10 11:46:56.047921 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 10 11:46:56.047921 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 10 11:46:56.050231 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:46:56.050231 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:46:56.050231 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 10 11:46:56.050231 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 10 11:46:56.050231 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 10 11:46:56.050231 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 10 11:46:56.050231 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:46:56.050231 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:46:56.050231 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:46:56.050231 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:46:56.050231 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:46:56.050231 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application with Lock( Oct 10 11:46:56.050231 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:46:56.050231 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:46:56.050231 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:46:56.050231 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:46:56.050231 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:46:56.050231 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:46:56.052918 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:46:56.052918 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:46:56.052918 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 10 11:46:56.052918 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 10 11:46:56.052918 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 10 11:46:56.052918 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 10 11:46:56.052918 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 10 11:46:56.052918 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 10 11:46:56.052918 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Oct 10 11:46:56.052918 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Oct 10 11:46:56.054901 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 304/580] 10.222.0.46 () {56 vars in 1102 bytes} [Mon Oct 10 11:46:55 2022] GET /identity/v3/users/neutron => generated 85 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:46:56.057941 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b097cdd8-2da4-4c9f-9dd9-435782f10d81 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:56.067357 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748a93ce-fe20-471d-baa6-37307d580d34 None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:56.114613 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5f70ee-3450-4094-962b-7199bd771faf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:56.115308 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5f70ee-3450-4094-962b-7199bd771faf None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:56.115591 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5f70ee-3450-4094-962b-7199bd771faf None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:56.115862 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5f70ee-3450-4094-962b-7199bd771faf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:56.130459 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5f70ee-3450-4094-962b-7199bd771faf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:56.132081 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5f70ee-3450-4094-962b-7199bd771faf None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:56.153360 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application [None req-7a5f70ee-3450-4094-962b-7199bd771faf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:46:56.153360 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:46:56.153360 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:46:56.153360 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:46:56.153360 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:46:56.153360 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:46:56.153360 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:46:56.153360 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:46:56.153360 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:46:56.153360 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:46:56.153360 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:46:56.153360 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:46:56.153360 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 10 11:46:56.153360 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 10 11:46:56.153360 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 10 11:46:56.153360 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 10 11:46:56.155885 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:46:56.155885 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:46:56.155885 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:46:56.155885 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:46:56.155885 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:46:56.155885 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:46:56.155885 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:46:56.155885 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application with Lock( Oct 10 11:46:56.155885 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:46:56.155885 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:46:56.155885 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:46:56.155885 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:46:56.155885 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:46:56.155885 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:46:56.155885 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:46:56.155885 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:46:56.155885 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 10 11:46:56.157645 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 10 11:46:56.157645 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:46:56.157645 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Oct 10 11:46:56.157645 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 277/581] 10.222.0.46 () {56 vars in 1108 bytes} [Mon Oct 10 11:46:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:46:56.157645 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5f70ee-3450-4094-962b-7199bd771faf None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:56.234826 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b097cdd8-2da4-4c9f-9dd9-435782f10d81 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:56.254100 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:56.270500 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:56.291694 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc057b9-15fd-42a8-8cc9-d17d28673dbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:56.292579 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc057b9-15fd-42a8-8cc9-d17d28673dbe None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:56.293018 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc057b9-15fd-42a8-8cc9-d17d28673dbe None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:56.293468 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc057b9-15fd-42a8-8cc9-d17d28673dbe None admin] PATH_INFO: `/v3/projects` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:56.295789 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc057b9-15fd-42a8-8cc9-d17d28673dbe None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:46:56.296270 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc057b9-15fd-42a8-8cc9-d17d28673dbe None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:56.298951 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc057b9-15fd-42a8-8cc9-d17d28673dbe None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:56.320257 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 305/582] 10.222.0.46 () {56 vars in 1148 bytes} [Mon Oct 10 11:46:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:46:56.321837 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc057b9-15fd-42a8-8cc9-d17d28673dbe None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:56.338154 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5f70ee-3450-4094-962b-7199bd771faf None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:56.402369 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31dba5b2-83f0-4e7f-af45-339f3b55166c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:56.403504 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31dba5b2-83f0-4e7f-af45-339f3b55166c None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:56.404086 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31dba5b2-83f0-4e7f-af45-339f3b55166c None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:56.404676 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31dba5b2-83f0-4e7f-af45-339f3b55166c None admin] PATH_INFO: `/v3/roles` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:56.405906 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31dba5b2-83f0-4e7f-af45-339f3b55166c None admin] RBAC: Adding query filter params (name=service) {{(pid=81676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:46:56.406454 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31dba5b2-83f0-4e7f-af45-339f3b55166c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:56.409085 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31dba5b2-83f0-4e7f-af45-339f3b55166c None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:56.425325 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 278/583] 10.222.0.46 () {56 vars in 1103 bytes} [Mon Oct 10 11:46:56 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:46:56.426714 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31dba5b2-83f0-4e7f-af45-339f3b55166c None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:56.467907 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc057b9-15fd-42a8-8cc9-d17d28673dbe None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:56.556907 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13661e2d-b424-46bd-87f0-24d45fcc1d94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:56.557956 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13661e2d-b424-46bd-87f0-24d45fcc1d94 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:56.558453 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13661e2d-b424-46bd-87f0-24d45fcc1d94 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:56.558995 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13661e2d-b424-46bd-87f0-24d45fcc1d94 None admin] PATH_INFO: `/v3/users` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:56.560340 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13661e2d-b424-46bd-87f0-24d45fcc1d94 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:46:56.560899 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13661e2d-b424-46bd-87f0-24d45fcc1d94 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:56.563656 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13661e2d-b424-46bd-87f0-24d45fcc1d94 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:56.599505 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 306/584] 10.222.0.46 () {56 vars in 1139 bytes} [Mon Oct 10 11:46:56 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:46:56.600758 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13661e2d-b424-46bd-87f0-24d45fcc1d94 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:56.624748 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31dba5b2-83f0-4e7f-af45-339f3b55166c None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:56.655063 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:56.670216 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:56.678972 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7fb527-e37f-45f6-9628-b1c620177f22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:56.679607 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7fb527-e37f-45f6-9628-b1c620177f22 None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:56.679903 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7fb527-e37f-45f6-9628-b1c620177f22 None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:56.680163 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7fb527-e37f-45f6-9628-b1c620177f22 None admin] PATH_INFO: `/v3/domains` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:56.680825 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7fb527-e37f-45f6-9628-b1c620177f22 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:46:56.681131 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7fb527-e37f-45f6-9628-b1c620177f22 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:56.682640 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7fb527-e37f-45f6-9628-b1c620177f22 None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:56.704422 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 279/585] 10.222.0.46 () {56 vars in 1109 bytes} [Mon Oct 10 11:46:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:46:56.704422 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7fb527-e37f-45f6-9628-b1c620177f22 None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:56.796706 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13661e2d-b424-46bd-87f0-24d45fcc1d94 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:56.836729 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7fb527-e37f-45f6-9628-b1c620177f22 None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:56.859224 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58721680-85d3-4bf5-95e3-ba334f9ec55d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:56.860039 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58721680-85d3-4bf5-95e3-ba334f9ec55d None admin] REQUEST_METHOD: `PUT` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:56.860487 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58721680-85d3-4bf5-95e3-ba334f9ec55d None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:56.860934 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58721680-85d3-4bf5-95e3-ba334f9ec55d None admin] PATH_INFO: `/v3/projects/4ad3772f7b6a47c28b7ba37a9316a3c1/users/d3ae093470484b93b23dcfae1657df9e/roles/3c35be38787740f4834c728a2ce73b6b` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:56.886390 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58721680-85d3-4bf5-95e3-ba334f9ec55d None admin] RBAC: Authorizing `identity:create_grant(project_id=4ad3772f7b6a47c28b7ba37a9316a3c1, user_id=d3ae093470484b93b23dcfae1657df9e, role_id=3c35be38787740f4834c728a2ce73b6b)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:56.888885 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58721680-85d3-4bf5-95e3-ba334f9ec55d None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:56.895954 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9993d982-6f1e-4130-8f43-836563767fd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:56.896784 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9993d982-6f1e-4130-8f43-836563767fd2 None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:56.896965 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9993d982-6f1e-4130-8f43-836563767fd2 None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:56.897283 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9993d982-6f1e-4130-8f43-836563767fd2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:56.915110 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9993d982-6f1e-4130-8f43-836563767fd2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:56.916733 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9993d982-6f1e-4130-8f43-836563767fd2 None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:56.934568 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application [None req-9993d982-6f1e-4130-8f43-836563767fd2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:46:56.934568 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:46:56.934568 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:46:56.934568 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:46:56.934568 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:46:56.934568 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:46:56.934568 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:46:56.934568 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:46:56.934568 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:46:56.934568 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:46:56.934568 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:46:56.934568 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:46:56.934568 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 10 11:46:56.934568 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 10 11:46:56.934568 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 10 11:46:56.934568 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 10 11:46:56.936599 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:46:56.936599 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:46:56.936599 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:46:56.936599 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:46:56.936599 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:46:56.936599 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:46:56.936599 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:46:56.936599 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application with Lock( Oct 10 11:46:56.936599 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:46:56.936599 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:46:56.936599 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:46:56.936599 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:46:56.936599 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:46:56.936599 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:46:56.936599 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:46:56.936599 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:46:56.936599 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 10 11:46:56.938644 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 10 11:46:56.938644 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:46:56.938644 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Oct 10 11:46:56.941393 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 280/586] 10.222.0.46 () {56 vars in 1108 bytes} [Mon Oct 10 11:46:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:46:56.942451 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9993d982-6f1e-4130-8f43-836563767fd2 None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:56.974718 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 307/587] 10.222.0.46 () {58 vars in 1439 bytes} [Mon Oct 10 11:46:56 2022] PUT /identity/v3/projects/4ad3772f7b6a47c28b7ba37a9316a3c1/users/d3ae093470484b93b23dcfae1657df9e/roles/3c35be38787740f4834c728a2ce73b6b => generated 0 bytes in 367 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 11:46:56.974718 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58721680-85d3-4bf5-95e3-ba334f9ec55d None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:57.030150 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:57.126957 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:57.170007 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58721680-85d3-4bf5-95e3-ba334f9ec55d None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:57.175110 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9993d982-6f1e-4130-8f43-836563767fd2 None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:57.240909 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b07270-b44e-4b7f-a549-ac657851525b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:57.242452 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b07270-b44e-4b7f-a549-ac657851525b None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:57.242773 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b07270-b44e-4b7f-a549-ac657851525b None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:57.243190 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b07270-b44e-4b7f-a549-ac657851525b None admin] PATH_INFO: `/v3/users/nova` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:57.250766 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c14772-2d96-4f06-8f1e-86fade34164c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:57.251911 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c14772-2d96-4f06-8f1e-86fade34164c None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:57.252154 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b07270-b44e-4b7f-a549-ac657851525b None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:57.252396 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c14772-2d96-4f06-8f1e-86fade34164c None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:57.252872 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c14772-2d96-4f06-8f1e-86fade34164c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:57.253617 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b07270-b44e-4b7f-a549-ac657851525b None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:57.268920 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application [None req-16b07270-b44e-4b7f-a549-ac657851525b None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 10 11:46:57.268920 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:46:57.268920 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:46:57.268920 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:46:57.268920 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:46:57.268920 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:46:57.268920 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:46:57.268920 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:46:57.268920 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:46:57.268920 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:46:57.268920 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:46:57.268920 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:46:57.268920 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 10 11:46:57.268920 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 10 11:46:57.268920 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 10 11:46:57.268920 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 10 11:46:57.270633 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:46:57.270633 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:46:57.270633 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 10 11:46:57.270633 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 10 11:46:57.270633 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 10 11:46:57.270633 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 10 11:46:57.270633 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:46:57.270633 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:46:57.270633 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:46:57.270633 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:46:57.270633 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:46:57.270633 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application with Lock( Oct 10 11:46:57.270633 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:46:57.270633 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:46:57.270633 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:46:57.270633 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:46:57.270633 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:46:57.270633 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:46:57.273379 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:46:57.273379 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:46:57.273379 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 10 11:46:57.273379 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 10 11:46:57.273379 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 10 11:46:57.273379 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 10 11:46:57.273379 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 10 11:46:57.273379 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 10 11:46:57.273379 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 10 11:46:57.273379 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Oct 10 11:46:57.273379 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 308/588] 10.222.0.46 () {56 vars in 1093 bytes} [Mon Oct 10 11:46:56 2022] GET /identity/v3/users/nova => generated 82 bytes in 302 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:46:57.273379 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b07270-b44e-4b7f-a549-ac657851525b None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:57.280484 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c14772-2d96-4f06-8f1e-86fade34164c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:57.283755 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c14772-2d96-4f06-8f1e-86fade34164c None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:57.322761 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application [None req-57c14772-2d96-4f06-8f1e-86fade34164c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:46:57.322761 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:46:57.322761 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:46:57.322761 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:46:57.322761 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:46:57.322761 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:46:57.322761 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:46:57.322761 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:46:57.322761 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:46:57.322761 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:46:57.322761 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:46:57.322761 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:46:57.322761 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 10 11:46:57.322761 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 10 11:46:57.322761 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 10 11:46:57.322761 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 10 11:46:57.324563 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:46:57.324563 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:46:57.324563 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:46:57.324563 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:46:57.324563 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:46:57.324563 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:46:57.324563 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:46:57.324563 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application with Lock( Oct 10 11:46:57.324563 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:46:57.324563 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:46:57.324563 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:46:57.324563 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:46:57.324563 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:46:57.324563 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:46:57.324563 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:46:57.324563 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:46:57.324563 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 10 11:46:57.326468 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 10 11:46:57.326468 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:46:57.326468 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Oct 10 11:46:57.328465 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 281/589] 10.222.0.46 () {56 vars in 1108 bytes} [Mon Oct 10 11:46:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 386 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:46:57.334702 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c14772-2d96-4f06-8f1e-86fade34164c None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:57.452332 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b07270-b44e-4b7f-a549-ac657851525b None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:57.473942 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:57.496275 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c14772-2d96-4f06-8f1e-86fade34164c None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:57.503275 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f41545-9751-4c29-84cc-01b7961e4c6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:57.503912 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f41545-9751-4c29-84cc-01b7961e4c6d None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:57.504194 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f41545-9751-4c29-84cc-01b7961e4c6d None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:57.505517 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f41545-9751-4c29-84cc-01b7961e4c6d None admin] PATH_INFO: `/v3/domains` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:57.505517 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f41545-9751-4c29-84cc-01b7961e4c6d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:46:57.505517 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f41545-9751-4c29-84cc-01b7961e4c6d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:57.506928 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f41545-9751-4c29-84cc-01b7961e4c6d None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:57.519536 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:57.529192 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 309/590] 10.222.0.46 () {56 vars in 1109 bytes} [Mon Oct 10 11:46:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:46:57.530226 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f41545-9751-4c29-84cc-01b7961e4c6d None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:57.612269 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb56457-5ac6-43f1-9b1f-5d73e8a845dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:57.614907 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb56457-5ac6-43f1-9b1f-5d73e8a845dd None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:57.615734 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb56457-5ac6-43f1-9b1f-5d73e8a845dd None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:57.616541 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb56457-5ac6-43f1-9b1f-5d73e8a845dd None admin] PATH_INFO: `/v3/users` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:57.619127 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb56457-5ac6-43f1-9b1f-5d73e8a845dd None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:46:57.619453 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb56457-5ac6-43f1-9b1f-5d73e8a845dd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:57.621205 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb56457-5ac6-43f1-9b1f-5d73e8a845dd None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:57.642265 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 282/591] 10.222.0.46 () {56 vars in 1133 bytes} [Mon Oct 10 11:46:57 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:46:57.643180 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb56457-5ac6-43f1-9b1f-5d73e8a845dd None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:57.694001 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f41545-9751-4c29-84cc-01b7961e4c6d None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:57.756835 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb56457-5ac6-43f1-9b1f-5d73e8a845dd None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:57.769993 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d778d5-5288-4494-879a-e18031ecefe2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:57.771600 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d778d5-5288-4494-879a-e18031ecefe2 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:57.772505 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d778d5-5288-4494-879a-e18031ecefe2 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:57.773353 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d778d5-5288-4494-879a-e18031ecefe2 None admin] PATH_INFO: `/v3/domains` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:57.776279 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d778d5-5288-4494-879a-e18031ecefe2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:46:57.777278 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d778d5-5288-4494-879a-e18031ecefe2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:57.781401 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d778d5-5288-4494-879a-e18031ecefe2 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:57.797314 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244a3c60-5c1f-4bbc-b2c8-38696533dbf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:57.798009 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244a3c60-5c1f-4bbc-b2c8-38696533dbf1 None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:57.798493 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244a3c60-5c1f-4bbc-b2c8-38696533dbf1 None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:57.798812 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244a3c60-5c1f-4bbc-b2c8-38696533dbf1 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:57.808298 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244a3c60-5c1f-4bbc-b2c8-38696533dbf1 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:57.817610 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244a3c60-5c1f-4bbc-b2c8-38696533dbf1 None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:57.817828 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 310/592] 10.222.0.46 () {56 vars in 1109 bytes} [Mon Oct 10 11:46:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:46:57.817828 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d778d5-5288-4494-879a-e18031ecefe2 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:57.844333 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application [None req-244a3c60-5c1f-4bbc-b2c8-38696533dbf1 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Oct 10 11:46:57.844333 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:46:57.844333 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:46:57.844333 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:46:57.844333 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:46:57.844333 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:46:57.844333 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:46:57.844333 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:46:57.844333 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:46:57.844333 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:46:57.844333 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:46:57.844333 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:46:57.844333 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 10 11:46:57.844333 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 10 11:46:57.844333 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 10 11:46:57.844333 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 10 11:46:57.846832 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:46:57.846832 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:46:57.846832 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 10 11:46:57.846832 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 10 11:46:57.846832 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 10 11:46:57.846832 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 10 11:46:57.846832 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:46:57.846832 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:46:57.846832 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:46:57.846832 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:46:57.846832 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:46:57.846832 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application with Lock( Oct 10 11:46:57.846832 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:46:57.846832 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:46:57.846832 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:46:57.846832 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:46:57.846832 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:46:57.846832 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:46:57.848736 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:46:57.848736 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:46:57.848736 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 10 11:46:57.848736 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 10 11:46:57.848736 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 10 11:46:57.848736 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 10 11:46:57.848736 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 10 11:46:57.848736 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 10 11:46:57.848736 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Oct 10 11:46:57.848736 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Oct 10 11:46:57.848736 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 283/593] 10.222.0.46 () {56 vars in 1099 bytes} [Mon Oct 10 11:46:57 2022] GET /identity/v3/users/glance => generated 84 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:46:57.849896 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244a3c60-5c1f-4bbc-b2c8-38696533dbf1 None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:57.875300 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:57.934394 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:58.020345 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d778d5-5288-4494-879a-e18031ecefe2 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:58.057612 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244a3c60-5c1f-4bbc-b2c8-38696533dbf1 None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:58.095681 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be7a572-d597-4313-a125-f0edc2b46150 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:58.097043 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be7a572-d597-4313-a125-f0edc2b46150 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:58.097697 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be7a572-d597-4313-a125-f0edc2b46150 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:58.098360 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be7a572-d597-4313-a125-f0edc2b46150 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:58.121145 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be7a572-d597-4313-a125-f0edc2b46150 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:58.124216 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be7a572-d597-4313-a125-f0edc2b46150 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:58.129549 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2912f521-599c-46aa-a1b7-b612c7fe8e24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:58.130793 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2912f521-599c-46aa-a1b7-b612c7fe8e24 None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:58.131428 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2912f521-599c-46aa-a1b7-b612c7fe8e24 None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:58.131998 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2912f521-599c-46aa-a1b7-b612c7fe8e24 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:58.145598 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2912f521-599c-46aa-a1b7-b612c7fe8e24 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:58.148598 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2912f521-599c-46aa-a1b7-b612c7fe8e24 None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:58.156010 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application [None req-7be7a572-d597-4313-a125-f0edc2b46150 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:46:58.156010 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:46:58.156010 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:46:58.156010 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:46:58.156010 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:46:58.156010 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:46:58.156010 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:46:58.156010 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:46:58.156010 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:46:58.156010 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:46:58.156010 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:46:58.156010 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:46:58.156010 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 10 11:46:58.156010 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 10 11:46:58.156010 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 10 11:46:58.156010 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 10 11:46:58.158642 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:46:58.158642 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:46:58.158642 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:46:58.158642 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:46:58.158642 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:46:58.158642 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:46:58.158642 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:46:58.158642 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application with Lock( Oct 10 11:46:58.158642 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:46:58.158642 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:46:58.158642 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:46:58.158642 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:46:58.158642 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:46:58.158642 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:46:58.158642 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:46:58.158642 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:46:58.158642 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 10 11:46:58.160670 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 10 11:46:58.160670 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:46:58.160670 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Oct 10 11:46:58.161957 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 311/594] 10.222.0.46 () {56 vars in 1108 bytes} [Mon Oct 10 11:46:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 349 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:46:58.163610 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be7a572-d597-4313-a125-f0edc2b46150 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:58.181476 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application [None req-2912f521-599c-46aa-a1b7-b612c7fe8e24 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 10 11:46:58.181476 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:46:58.181476 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:46:58.181476 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:46:58.181476 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:46:58.181476 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:46:58.181476 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:46:58.181476 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:46:58.181476 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:46:58.181476 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:46:58.181476 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:46:58.181476 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:46:58.181476 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 10 11:46:58.181476 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 10 11:46:58.181476 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 10 11:46:58.183544 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 10 11:46:58.183544 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:46:58.183544 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:46:58.183544 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:46:58.183544 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:46:58.183544 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:46:58.183544 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:46:58.183544 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:46:58.183544 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application with Lock( Oct 10 11:46:58.183544 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:46:58.183544 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:46:58.183544 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:46:58.183544 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:46:58.183544 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:46:58.183544 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:46:58.183544 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:46:58.183544 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:46:58.185384 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 10 11:46:58.185384 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 10 11:46:58.185384 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 10 11:46:58.185384 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 10 11:46:58.185384 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 10 11:46:58.185384 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 10 11:46:58.185384 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 10 11:46:58.185384 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Oct 10 11:46:58.189397 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 284/595] 10.222.0.46 () {56 vars in 1111 bytes} [Mon Oct 10 11:46:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 341 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:46:58.191249 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2912f521-599c-46aa-a1b7-b612c7fe8e24 None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:58.339413 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:58.378917 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be7a572-d597-4313-a125-f0edc2b46150 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:58.389257 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2912f521-599c-46aa-a1b7-b612c7fe8e24 None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:58.400139 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:58.434859 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114f899d-7066-4fdd-9ba8-3c672406090f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:58.436402 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114f899d-7066-4fdd-9ba8-3c672406090f None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:58.436824 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114f899d-7066-4fdd-9ba8-3c672406090f None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:58.437264 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114f899d-7066-4fdd-9ba8-3c672406090f None admin] PATH_INFO: `/v3/domains` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:58.437468 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd970cc-f4ba-4a77-bc17-5d8d7bc7ea80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:58.438153 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd970cc-f4ba-4a77-bc17-5d8d7bc7ea80 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:58.438402 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114f899d-7066-4fdd-9ba8-3c672406090f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:46:58.438510 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd970cc-f4ba-4a77-bc17-5d8d7bc7ea80 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:58.438944 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd970cc-f4ba-4a77-bc17-5d8d7bc7ea80 None admin] PATH_INFO: `/v3/users` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:58.439139 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114f899d-7066-4fdd-9ba8-3c672406090f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:58.439864 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd970cc-f4ba-4a77-bc17-5d8d7bc7ea80 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:46:58.440254 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd970cc-f4ba-4a77-bc17-5d8d7bc7ea80 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:58.441061 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114f899d-7066-4fdd-9ba8-3c672406090f None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:58.442377 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd970cc-f4ba-4a77-bc17-5d8d7bc7ea80 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:58.460924 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 285/596] 10.222.0.46 () {56 vars in 1109 bytes} [Mon Oct 10 11:46:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:46:58.462097 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114f899d-7066-4fdd-9ba8-3c672406090f None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:58.484818 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 312/597] 10.222.0.46 () {56 vars in 1137 bytes} [Mon Oct 10 11:46:58 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:46:58.486553 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73fcf98-3943-442d-877c-bf3767b2beb4 None None] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:58.487019 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73fcf98-3943-442d-877c-bf3767b2beb4 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:58.487470 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73fcf98-3943-442d-877c-bf3767b2beb4 None None] PATH_INFO: `/` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:58.489176 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 313/598] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 10 11:46:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:46:58.490104 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73fcf98-3943-442d-877c-bf3767b2beb4 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:58.615591 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73fcf98-3943-442d-877c-bf3767b2beb4 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:58.632942 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114f899d-7066-4fdd-9ba8-3c672406090f None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:58.669130 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:58.674934 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7806fd-b931-45a3-b4f6-053d6be1cb3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:58.675742 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7806fd-b931-45a3-b4f6-053d6be1cb3c None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:58.676208 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7806fd-b931-45a3-b4f6-053d6be1cb3c None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:58.676643 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7806fd-b931-45a3-b4f6-053d6be1cb3c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:58.679045 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad64b9b9-7756-43ba-b997-22db31f05540 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:58.680105 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad64b9b9-7756-43ba-b997-22db31f05540 None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:58.680596 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad64b9b9-7756-43ba-b997-22db31f05540 None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:58.681104 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad64b9b9-7756-43ba-b997-22db31f05540 None admin] PATH_INFO: `/v3/projects` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:58.684687 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad64b9b9-7756-43ba-b997-22db31f05540 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:46:58.684687 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad64b9b9-7756-43ba-b997-22db31f05540 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:58.685630 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad64b9b9-7756-43ba-b997-22db31f05540 None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:58.686753 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7806fd-b931-45a3-b4f6-053d6be1cb3c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:58.688651 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7806fd-b931-45a3-b4f6-053d6be1cb3c None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:58.701728 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application [None req-5c7806fd-b931-45a3-b4f6-053d6be1cb3c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 10 11:46:58.701728 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:46:58.701728 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:46:58.701728 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:46:58.701728 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:46:58.701728 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:46:58.701728 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:46:58.701728 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:46:58.701728 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:46:58.701728 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:46:58.701728 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:46:58.701728 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:46:58.701728 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 10 11:46:58.701728 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 10 11:46:58.701728 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 10 11:46:58.702707 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 10 11:46:58.702707 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:46:58.702707 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:46:58.702707 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:46:58.702707 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:46:58.702707 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:46:58.702707 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:46:58.702707 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:46:58.702707 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application with Lock( Oct 10 11:46:58.702707 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:46:58.702707 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:46:58.702707 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:46:58.702707 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:46:58.702707 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:46:58.702707 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:46:58.702707 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:46:58.702707 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:46:58.703592 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 10 11:46:58.703592 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 10 11:46:58.703592 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 10 11:46:58.703592 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 10 11:46:58.703592 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 10 11:46:58.703592 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 10 11:46:58.703592 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 10 11:46:58.703592 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Oct 10 11:46:58.703592 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 314/599] 10.222.0.46 () {56 vars in 1111 bytes} [Mon Oct 10 11:46:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:46:58.704073 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7806fd-b931-45a3-b4f6-053d6be1cb3c None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:58.722166 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 286/600] 10.222.0.46 () {56 vars in 1148 bytes} [Mon Oct 10 11:46:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:46:58.722166 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd94a6f2-c5c0-4a8d-b678-ea973607b005 None None] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:58.722166 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd94a6f2-c5c0-4a8d-b678-ea973607b005 None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:58.722166 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd94a6f2-c5c0-4a8d-b678-ea973607b005 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:58.741919 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.auth.core [None req-cd94a6f2-c5c0-4a8d-b678-ea973607b005 None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:46:58.743152 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-cd94a6f2-c5c0-4a8d-b678-ea973607b005 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:46:58.750565 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:58.832404 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 287/601] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 10 11:46:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 10 11:46:58.832404 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd94a6f2-c5c0-4a8d-b678-ea973607b005 None None] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:58.867771 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7806fd-b931-45a3-b4f6-053d6be1cb3c None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:58.925576 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5637d1-c640-426b-ac98-df5227065100 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:58.926484 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5637d1-c640-426b-ac98-df5227065100 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:58.927009 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5637d1-c640-426b-ac98-df5227065100 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:58.927429 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5637d1-c640-426b-ac98-df5227065100 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:58.941444 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5637d1-c640-426b-ac98-df5227065100 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:58.943313 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5637d1-c640-426b-ac98-df5227065100 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:58.962902 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application [None req-0f5637d1-c640-426b-ac98-df5227065100 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:46:58.962902 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:46:58.962902 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:46:58.962902 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:46:58.962902 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:46:58.962902 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:46:58.962902 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:46:58.962902 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:46:58.962902 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:46:58.962902 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:46:58.962902 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:46:58.962902 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:46:58.962902 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 10 11:46:58.962902 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 10 11:46:58.962902 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 10 11:46:58.962902 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 10 11:46:58.964013 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:46:58.964013 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:46:58.964013 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:46:58.964013 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:46:58.964013 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:46:58.964013 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:46:58.964013 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:46:58.964013 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application with Lock( Oct 10 11:46:58.964013 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:46:58.964013 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:46:58.964013 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:46:58.964013 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:46:58.964013 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:46:58.964013 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:46:58.964013 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:46:58.964013 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:46:58.964013 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 10 11:46:58.964874 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 10 11:46:58.964874 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:46:58.964874 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Oct 10 11:46:58.967092 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 315/602] 10.222.0.46 () {56 vars in 1108 bytes} [Mon Oct 10 11:46:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:46:58.968435 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5637d1-c640-426b-ac98-df5227065100 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:58.982475 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd94a6f2-c5c0-4a8d-b678-ea973607b005 None None] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:59.018359 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:59.037911 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701be8a0-b540-4ec8-b049-dd3a5cf19e43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:59.039405 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701be8a0-b540-4ec8-b049-dd3a5cf19e43 None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:59.039405 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701be8a0-b540-4ec8-b049-dd3a5cf19e43 None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:59.039405 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701be8a0-b540-4ec8-b049-dd3a5cf19e43 None admin] PATH_INFO: `/v3/projects` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:59.039852 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701be8a0-b540-4ec8-b049-dd3a5cf19e43 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:46:59.040177 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701be8a0-b540-4ec8-b049-dd3a5cf19e43 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:59.041756 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701be8a0-b540-4ec8-b049-dd3a5cf19e43 None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:59.055089 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 288/603] 10.222.0.46 () {56 vars in 1148 bytes} [Mon Oct 10 11:46:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:46:59.055970 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701be8a0-b540-4ec8-b049-dd3a5cf19e43 None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:59.080793 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:59.104984 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5637d1-c640-426b-ac98-df5227065100 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:59.152126 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e266ae2-b314-4406-891b-08c15667edd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:59.153209 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e266ae2-b314-4406-891b-08c15667edd4 None admin] REQUEST_METHOD: `PUT` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:59.153645 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e266ae2-b314-4406-891b-08c15667edd4 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:59.154156 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e266ae2-b314-4406-891b-08c15667edd4 None admin] PATH_INFO: `/v3/projects/4ad3772f7b6a47c28b7ba37a9316a3c1/users/58b1c7c85302421cba823ace4201861f/roles/3c35be38787740f4834c728a2ce73b6b` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:59.177976 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701be8a0-b540-4ec8-b049-dd3a5cf19e43 None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:59.180479 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e266ae2-b314-4406-891b-08c15667edd4 None admin] RBAC: Authorizing `identity:create_grant(project_id=4ad3772f7b6a47c28b7ba37a9316a3c1, user_id=58b1c7c85302421cba823ace4201861f, role_id=3c35be38787740f4834c728a2ce73b6b)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:59.183444 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e266ae2-b314-4406-891b-08c15667edd4 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:59.222614 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69df19f6-c7ed-47ba-a927-1ae83338b0a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:59.223528 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69df19f6-c7ed-47ba-a927-1ae83338b0a4 None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:59.223986 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69df19f6-c7ed-47ba-a927-1ae83338b0a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:59.224456 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69df19f6-c7ed-47ba-a927-1ae83338b0a4 None admin] PATH_INFO: `/v3/domains` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:59.225326 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69df19f6-c7ed-47ba-a927-1ae83338b0a4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:46:59.225881 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69df19f6-c7ed-47ba-a927-1ae83338b0a4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:59.227875 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69df19f6-c7ed-47ba-a927-1ae83338b0a4 None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:59.243975 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 289/604] 10.222.0.46 () {56 vars in 1109 bytes} [Mon Oct 10 11:46:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:46:59.244853 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69df19f6-c7ed-47ba-a927-1ae83338b0a4 None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:59.261684 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 316/605] 10.222.0.46 () {58 vars in 1439 bytes} [Mon Oct 10 11:46:58 2022] PUT /identity/v3/projects/4ad3772f7b6a47c28b7ba37a9316a3c1/users/58b1c7c85302421cba823ace4201861f/roles/3c35be38787740f4834c728a2ce73b6b => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 11:46:59.263728 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e266ae2-b314-4406-891b-08c15667edd4 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:59.335837 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:59.477474 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69df19f6-c7ed-47ba-a927-1ae83338b0a4 None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:59.491435 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e266ae2-b314-4406-891b-08c15667edd4 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:59.552047 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:46:59.565114 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a929e57-64fd-4611-a3a6-168195fa9df8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:59.565824 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b341c2b6-bc36-4282-be11-a4c23d75a906 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:59.566915 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b341c2b6-bc36-4282-be11-a4c23d75a906 None admin] REQUEST_METHOD: `PUT` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:59.567427 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b341c2b6-bc36-4282-be11-a4c23d75a906 None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:59.567853 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a929e57-64fd-4611-a3a6-168195fa9df8 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:59.568017 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b341c2b6-bc36-4282-be11-a4c23d75a906 None admin] PATH_INFO: `/v3/projects/4ad3772f7b6a47c28b7ba37a9316a3c1/users/5f130bbb69e546ce99b4e14de81d7e54/roles/3c35be38787740f4834c728a2ce73b6b` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:59.568928 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a929e57-64fd-4611-a3a6-168195fa9df8 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:59.569802 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a929e57-64fd-4611-a3a6-168195fa9df8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:59.586896 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a929e57-64fd-4611-a3a6-168195fa9df8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:59.590936 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a929e57-64fd-4611-a3a6-168195fa9df8 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:59.597354 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b341c2b6-bc36-4282-be11-a4c23d75a906 None admin] RBAC: Authorizing `identity:create_grant(project_id=4ad3772f7b6a47c28b7ba37a9316a3c1, user_id=5f130bbb69e546ce99b4e14de81d7e54, role_id=3c35be38787740f4834c728a2ce73b6b)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:46:59.599869 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b341c2b6-bc36-4282-be11-a4c23d75a906 None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:46:59.619028 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application [None req-3a929e57-64fd-4611-a3a6-168195fa9df8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 10 11:46:59.619028 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:46:59.619028 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:46:59.619028 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:46:59.619028 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:46:59.619028 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:46:59.619028 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:46:59.619028 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:46:59.619028 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:46:59.619028 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:46:59.619028 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:46:59.619028 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:46:59.619028 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 10 11:46:59.619028 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 10 11:46:59.619028 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 10 11:46:59.624519 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 10 11:46:59.624519 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:46:59.624519 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:46:59.624519 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:46:59.624519 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:46:59.624519 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:46:59.624519 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:46:59.624519 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:46:59.624519 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application with Lock( Oct 10 11:46:59.624519 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:46:59.624519 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:46:59.624519 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:46:59.624519 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:46:59.624519 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:46:59.624519 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:46:59.624519 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:46:59.624519 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:46:59.626390 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 10 11:46:59.626390 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 10 11:46:59.626390 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 10 11:46:59.626390 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 10 11:46:59.626390 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 10 11:46:59.626390 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 10 11:46:59.626390 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 10 11:46:59.626390 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Oct 10 11:46:59.626390 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 317/606] 10.222.0.46 () {56 vars in 1111 bytes} [Mon Oct 10 11:46:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 359 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:46:59.626390 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a929e57-64fd-4611-a3a6-168195fa9df8 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:59.682868 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 290/607] 10.222.0.46 () {58 vars in 1439 bytes} [Mon Oct 10 11:46:59 2022] PUT /identity/v3/projects/4ad3772f7b6a47c28b7ba37a9316a3c1/users/5f130bbb69e546ce99b4e14de81d7e54/roles/3c35be38787740f4834c728a2ce73b6b => generated 0 bytes in 432 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 11:46:59.684774 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b341c2b6-bc36-4282-be11-a4c23d75a906 None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:46:59.835394 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a929e57-64fd-4611-a3a6-168195fa9df8 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:46:59.907449 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1804191d-b1f9-4b96-8e64-26b217298dd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:46:59.909261 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1804191d-b1f9-4b96-8e64-26b217298dd6 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:46:59.909971 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1804191d-b1f9-4b96-8e64-26b217298dd6 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:46:59.910635 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1804191d-b1f9-4b96-8e64-26b217298dd6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:46:59.931816 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1804191d-b1f9-4b96-8e64-26b217298dd6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 10 11:46:59.943132 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application [None req-1804191d-b1f9-4b96-8e64-26b217298dd6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 10 11:46:59.943132 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:46:59.943132 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:46:59.943132 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:46:59.943132 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:46:59.943132 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:46:59.943132 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:46:59.943132 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:46:59.943132 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:46:59.943132 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:46:59.943132 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:46:59.943132 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:46:59.943132 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 10 11:46:59.943132 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 10 11:46:59.943132 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 10 11:46:59.943132 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 10 11:46:59.944908 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 10 11:46:59.944908 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 10 11:46:59.944908 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 10 11:46:59.944908 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 10 11:46:59.944908 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:46:59.944908 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:46:59.944908 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:46:59.944908 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:46:59.944908 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:46:59.944908 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:46:59.944908 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:46:59.944908 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application with Lock( Oct 10 11:46:59.944908 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:46:59.944908 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:46:59.944908 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:46:59.944908 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:46:59.946569 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:46:59.946569 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:46:59.946569 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:46:59.946569 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:46:59.946569 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 10 11:46:59.946569 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 10 11:46:59.946569 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 10 11:46:59.946569 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 10 11:46:59.946569 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 10 11:46:59.946569 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 10 11:46:59.946569 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 10 11:46:59.946569 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Oct 10 11:46:59.951000 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 318/608] 10.222.0.46 () {56 vars in 1102 bytes} [Mon Oct 10 11:46:59 2022] GET /identity/v3/roles/service => generated 85 bytes in 329 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:46:59.970443 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1804191d-b1f9-4b96-8e64-26b217298dd6 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:00.012585 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:00.024617 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b341c2b6-bc36-4282-be11-a4c23d75a906 None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:00.104402 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:00.132768 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfab19e-bd3c-45d9-9ed0-59f57bf9a392 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:00.134275 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfab19e-bd3c-45d9-9ed0-59f57bf9a392 None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:00.135092 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfab19e-bd3c-45d9-9ed0-59f57bf9a392 None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:00.135832 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfab19e-bd3c-45d9-9ed0-59f57bf9a392 None admin] PATH_INFO: `/v3/projects` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:00.137356 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bfab19e-bd3c-45d9-9ed0-59f57bf9a392 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:47:00.138147 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bfab19e-bd3c-45d9-9ed0-59f57bf9a392 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:00.150732 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bfab19e-bd3c-45d9-9ed0-59f57bf9a392 None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:00.182272 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 291/609] 10.222.0.46 () {56 vars in 1148 bytes} [Mon Oct 10 11:46:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 498 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:47:00.184772 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ca6581-3c69-4ad2-b05e-0a49819a779e None None] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:00.185107 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ca6581-3c69-4ad2-b05e-0a49819a779e None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:00.185592 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ca6581-3c69-4ad2-b05e-0a49819a779e None None] PATH_INFO: `/` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:00.190814 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 292/610] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 10 11:47:00 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:47:00.195677 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ca6581-3c69-4ad2-b05e-0a49819a779e None None] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:00.206026 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1804191d-b1f9-4b96-8e64-26b217298dd6 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:00.247368 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4deee05-1a29-4ce4-8bc5-b87798d95032 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:00.248082 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4deee05-1a29-4ce4-8bc5-b87798d95032 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:00.249340 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4deee05-1a29-4ce4-8bc5-b87798d95032 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:00.249756 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4deee05-1a29-4ce4-8bc5-b87798d95032 None admin] PATH_INFO: `/v3/roles` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:00.250520 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4deee05-1a29-4ce4-8bc5-b87798d95032 None admin] RBAC: Adding query filter params (name=service) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:47:00.250862 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4deee05-1a29-4ce4-8bc5-b87798d95032 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:00.252300 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4deee05-1a29-4ce4-8bc5-b87798d95032 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:00.263116 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 319/611] 10.222.0.46 () {56 vars in 1103 bytes} [Mon Oct 10 11:46:59 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 11:47:00.264511 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7462ccbe-a3ef-4215-a368-701c76cf3086 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:00.264807 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7462ccbe-a3ef-4215-a368-701c76cf3086 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:00.265087 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7462ccbe-a3ef-4215-a368-701c76cf3086 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:00.286547 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-7462ccbe-a3ef-4215-a368-701c76cf3086 None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:47:00.287844 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-7462ccbe-a3ef-4215-a368-701c76cf3086 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:47:00.378735 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 320/612] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 10 11:47:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 10 11:47:00.379780 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7462ccbe-a3ef-4215-a368-701c76cf3086 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:00.418005 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ca6581-3c69-4ad2-b05e-0a49819a779e None None] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:00.529477 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57f42a1-72c9-4db6-9afa-539374616733 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:00.531298 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57f42a1-72c9-4db6-9afa-539374616733 None admin] REQUEST_METHOD: `PUT` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:00.531848 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57f42a1-72c9-4db6-9afa-539374616733 None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:00.532320 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57f42a1-72c9-4db6-9afa-539374616733 None admin] PATH_INFO: `/v3/projects/4ad3772f7b6a47c28b7ba37a9316a3c1/users/124f4aa8ab244ca2aadf4f2614df51e7/roles/3c35be38787740f4834c728a2ce73b6b` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:00.543065 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:00.563325 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57f42a1-72c9-4db6-9afa-539374616733 None admin] RBAC: Authorizing `identity:create_grant(project_id=4ad3772f7b6a47c28b7ba37a9316a3c1, user_id=124f4aa8ab244ca2aadf4f2614df51e7, role_id=3c35be38787740f4834c728a2ce73b6b)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:00.566114 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57f42a1-72c9-4db6-9afa-539374616733 None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:00.647961 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:00.657947 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 293/613] 10.222.0.46 () {58 vars in 1439 bytes} [Mon Oct 10 11:47:00 2022] PUT /identity/v3/projects/4ad3772f7b6a47c28b7ba37a9316a3c1/users/124f4aa8ab244ca2aadf4f2614df51e7/roles/3c35be38787740f4834c728a2ce73b6b => generated 0 bytes in 465 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 10 11:47:00.687426 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7462ccbe-a3ef-4215-a368-701c76cf3086 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:00.691538 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57f42a1-72c9-4db6-9afa-539374616733 None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:00.780157 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345d7b21-ebaf-404d-9431-f1d7dfd7f096 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:00.780823 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345d7b21-ebaf-404d-9431-f1d7dfd7f096 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:00.781078 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345d7b21-ebaf-404d-9431-f1d7dfd7f096 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:00.781359 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345d7b21-ebaf-404d-9431-f1d7dfd7f096 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:00.803764 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345d7b21-ebaf-404d-9431-f1d7dfd7f096 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:00.805260 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345d7b21-ebaf-404d-9431-f1d7dfd7f096 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:00.834461 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application [None req-345d7b21-ebaf-404d-9431-f1d7dfd7f096 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:47:00.834461 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:47:00.834461 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:47:00.834461 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:47:00.834461 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:47:00.834461 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:47:00.834461 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:47:00.834461 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:47:00.834461 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:47:00.834461 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:47:00.834461 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:47:00.834461 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:47:00.834461 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 10 11:47:00.834461 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 10 11:47:00.834461 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 10 11:47:00.834461 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 10 11:47:00.837854 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:47:00.837854 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:47:00.837854 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:47:00.837854 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:47:00.837854 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:47:00.837854 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:47:00.837854 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:47:00.837854 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application with Lock( Oct 10 11:47:00.837854 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:47:00.837854 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:47:00.837854 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:47:00.837854 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:47:00.837854 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:47:00.837854 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:47:00.837854 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:47:00.837854 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:47:00.837854 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 10 11:47:00.841088 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 10 11:47:00.841088 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:47:00.841088 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Oct 10 11:47:00.841088 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 321/614] 10.222.0.46 () {56 vars in 1108 bytes} [Mon Oct 10 11:47:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 457 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:47:00.846082 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345d7b21-ebaf-404d-9431-f1d7dfd7f096 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:00.916275 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57f42a1-72c9-4db6-9afa-539374616733 None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:00.998878 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a089e2-44f5-4e29-80eb-91c06927b87a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:00.999900 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a089e2-44f5-4e29-80eb-91c06927b87a None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:01.000359 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a089e2-44f5-4e29-80eb-91c06927b87a None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:01.000815 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a089e2-44f5-4e29-80eb-91c06927b87a None admin] PATH_INFO: `/v3/roles/service` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:01.030321 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:01.031902 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a089e2-44f5-4e29-80eb-91c06927b87a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 10 11:47:01.038048 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345d7b21-ebaf-404d-9431-f1d7dfd7f096 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:01.044354 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application [None req-f7a089e2-44f5-4e29-80eb-91c06927b87a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 10 11:47:01.044354 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:47:01.044354 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:47:01.044354 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:47:01.044354 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:47:01.044354 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:47:01.044354 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:47:01.044354 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:47:01.044354 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:47:01.044354 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:47:01.044354 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:47:01.044354 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:47:01.044354 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 10 11:47:01.044354 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 10 11:47:01.044354 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 10 11:47:01.044354 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 10 11:47:01.050296 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 10 11:47:01.050296 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 10 11:47:01.050296 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 10 11:47:01.050296 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 10 11:47:01.050296 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:47:01.050296 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:47:01.050296 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:47:01.050296 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:47:01.050296 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:47:01.050296 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:47:01.050296 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:47:01.050296 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application with Lock( Oct 10 11:47:01.050296 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:47:01.050296 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:47:01.050296 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:47:01.050296 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:47:01.052015 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:47:01.052015 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:47:01.052015 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:47:01.052015 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:47:01.052015 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 10 11:47:01.052015 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 10 11:47:01.052015 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 10 11:47:01.052015 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 10 11:47:01.052015 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 10 11:47:01.052015 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 10 11:47:01.052015 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 10 11:47:01.052015 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Oct 10 11:47:01.052015 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 294/615] 10.222.0.46 () {56 vars in 1102 bytes} [Mon Oct 10 11:47:00 2022] GET /identity/v3/roles/service => generated 85 bytes in 358 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 10 11:47:01.056699 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a089e2-44f5-4e29-80eb-91c06927b87a None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:01.093334 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4b8fe8-5d83-4c95-afb4-80edc3437510 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:01.093967 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4b8fe8-5d83-4c95-afb4-80edc3437510 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:01.094249 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4b8fe8-5d83-4c95-afb4-80edc3437510 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:01.094524 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4b8fe8-5d83-4c95-afb4-80edc3437510 None admin] PATH_INFO: `/v3/domains` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:01.095215 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4b8fe8-5d83-4c95-afb4-80edc3437510 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:47:01.095522 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4b8fe8-5d83-4c95-afb4-80edc3437510 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:01.096998 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4b8fe8-5d83-4c95-afb4-80edc3437510 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:01.112220 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 322/616] 10.222.0.46 () {56 vars in 1109 bytes} [Mon Oct 10 11:47:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:47:01.124120 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4b8fe8-5d83-4c95-afb4-80edc3437510 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:01.209555 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:01.290981 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a089e2-44f5-4e29-80eb-91c06927b87a None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:01.315350 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4b8fe8-5d83-4c95-afb4-80edc3437510 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:01.367095 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b460f2d-ce66-47fe-992f-7fcfa12ca8a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:01.367725 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b460f2d-ce66-47fe-992f-7fcfa12ca8a1 None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:01.368009 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b460f2d-ce66-47fe-992f-7fcfa12ca8a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:01.368284 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b460f2d-ce66-47fe-992f-7fcfa12ca8a1 None admin] PATH_INFO: `/v3/roles` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:01.369554 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b460f2d-ce66-47fe-992f-7fcfa12ca8a1 None admin] RBAC: Adding query filter params (name=service) {{(pid=81676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:47:01.370388 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b460f2d-ce66-47fe-992f-7fcfa12ca8a1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:01.372581 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b460f2d-ce66-47fe-992f-7fcfa12ca8a1 None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:01.376323 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83e895f-3b89-498d-8855-c0de3702ac0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:01.376976 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83e895f-3b89-498d-8855-c0de3702ac0e None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:01.377317 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83e895f-3b89-498d-8855-c0de3702ac0e None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:01.377597 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83e895f-3b89-498d-8855-c0de3702ac0e None admin] PATH_INFO: `/v3/users/swift` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:01.392764 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83e895f-3b89-498d-8855-c0de3702ac0e None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:01.394360 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83e895f-3b89-498d-8855-c0de3702ac0e None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:01.395321 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 295/617] 10.222.0.46 () {56 vars in 1103 bytes} [Mon Oct 10 11:47:01 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 340 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 11:47:01.402460 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b460f2d-ce66-47fe-992f-7fcfa12ca8a1 None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:01.424234 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application [None req-a83e895f-3b89-498d-8855-c0de3702ac0e None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 10 11:47:01.424234 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:47:01.424234 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:47:01.424234 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:47:01.424234 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:47:01.424234 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:47:01.424234 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:47:01.424234 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:47:01.424234 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:47:01.424234 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:47:01.424234 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:47:01.424234 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:47:01.424234 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 10 11:47:01.424234 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 10 11:47:01.424234 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 10 11:47:01.424234 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 10 11:47:01.428001 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:47:01.428001 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:47:01.428001 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 10 11:47:01.428001 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 10 11:47:01.428001 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 10 11:47:01.428001 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 10 11:47:01.428001 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:47:01.428001 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:47:01.428001 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:47:01.428001 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:47:01.428001 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:47:01.428001 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application with Lock( Oct 10 11:47:01.428001 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:47:01.428001 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:47:01.428001 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:47:01.428001 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:47:01.428001 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:47:01.428001 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:47:01.431110 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:47:01.431110 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:47:01.431110 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 10 11:47:01.431110 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 10 11:47:01.431110 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 10 11:47:01.431110 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 10 11:47:01.431110 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 10 11:47:01.431110 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 10 11:47:01.431110 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 10 11:47:01.431110 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Oct 10 11:47:01.431110 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 323/618] 10.222.0.46 () {56 vars in 1096 bytes} [Mon Oct 10 11:47:01 2022] GET /identity/v3/users/swift => generated 83 bytes in 302 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:47:01.436703 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83e895f-3b89-498d-8855-c0de3702ac0e None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:01.484324 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:01.589605 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b460f2d-ce66-47fe-992f-7fcfa12ca8a1 None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:01.660413 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83e895f-3b89-498d-8855-c0de3702ac0e None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:01.672713 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:01.691607 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1206a6b-57f7-4430-9d76-e0c7d9704629 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:01.692452 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1206a6b-57f7-4430-9d76-e0c7d9704629 None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:01.692794 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1206a6b-57f7-4430-9d76-e0c7d9704629 None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:01.693084 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1206a6b-57f7-4430-9d76-e0c7d9704629 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:01.718292 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1206a6b-57f7-4430-9d76-e0c7d9704629 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:01.719821 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1206a6b-57f7-4430-9d76-e0c7d9704629 None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:01.729283 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e3834e-6d6e-4324-9ad6-243fabb3550b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:01.729923 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e3834e-6d6e-4324-9ad6-243fabb3550b None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:01.730207 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e3834e-6d6e-4324-9ad6-243fabb3550b None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:01.730481 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e3834e-6d6e-4324-9ad6-243fabb3550b None admin] PATH_INFO: `/v3/users` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:01.731277 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e3834e-6d6e-4324-9ad6-243fabb3550b None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:47:01.731584 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e3834e-6d6e-4324-9ad6-243fabb3550b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:01.733097 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e3834e-6d6e-4324-9ad6-243fabb3550b None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:01.753716 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application [None req-e1206a6b-57f7-4430-9d76-e0c7d9704629 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:47:01.753716 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:47:01.753716 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:47:01.753716 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:47:01.753716 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:47:01.753716 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:47:01.753716 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:47:01.753716 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:47:01.753716 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:47:01.753716 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:47:01.753716 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:47:01.753716 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:47:01.753716 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 10 11:47:01.753716 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 10 11:47:01.753716 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 10 11:47:01.753716 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 10 11:47:01.755586 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:47:01.755586 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:47:01.755586 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:47:01.755586 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:47:01.755586 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:47:01.755586 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:47:01.755586 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:47:01.755586 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application with Lock( Oct 10 11:47:01.755586 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:47:01.755586 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:47:01.755586 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:47:01.755586 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:47:01.755586 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:47:01.755586 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:47:01.755586 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:47:01.755586 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:47:01.755586 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 10 11:47:01.757307 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 10 11:47:01.757307 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:47:01.757307 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Oct 10 11:47:01.762913 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 296/619] 10.222.0.46 () {56 vars in 1108 bytes} [Mon Oct 10 11:47:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 356 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:47:01.780545 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1206a6b-57f7-4430-9d76-e0c7d9704629 None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:01.800150 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 324/620] 10.222.0.46 () {56 vars in 1135 bytes} [Mon Oct 10 11:47:01 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 363 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:47:01.807823 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e3834e-6d6e-4324-9ad6-243fabb3550b None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:01.979327 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1206a6b-57f7-4430-9d76-e0c7d9704629 None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:02.052116 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786b5df1-eb14-460d-bb90-0b65a3f0e5ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:02.053616 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786b5df1-eb14-460d-bb90-0b65a3f0e5ed None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:02.055683 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786b5df1-eb14-460d-bb90-0b65a3f0e5ed None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:02.056442 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786b5df1-eb14-460d-bb90-0b65a3f0e5ed None admin] PATH_INFO: `/v3/domains` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:02.058223 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786b5df1-eb14-460d-bb90-0b65a3f0e5ed None admin] RBAC: Adding query filter params (name=Default) {{(pid=81676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:47:02.061028 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786b5df1-eb14-460d-bb90-0b65a3f0e5ed None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:02.064011 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786b5df1-eb14-460d-bb90-0b65a3f0e5ed None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:02.109726 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 297/621] 10.222.0.46 () {56 vars in 1109 bytes} [Mon Oct 10 11:47:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 331 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:47:02.111943 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:02.118198 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786b5df1-eb14-460d-bb90-0b65a3f0e5ed None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:02.207463 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e3834e-6d6e-4324-9ad6-243fabb3550b None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:02.238487 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:02.331715 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e004b30-24b3-4e79-b777-b0dbfff033f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:02.332670 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e004b30-24b3-4e79-b777-b0dbfff033f2 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:02.333159 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e004b30-24b3-4e79-b777-b0dbfff033f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:02.333632 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e004b30-24b3-4e79-b777-b0dbfff033f2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:02.366991 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e004b30-24b3-4e79-b777-b0dbfff033f2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:02.369978 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e004b30-24b3-4e79-b777-b0dbfff033f2 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:02.423093 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application [None req-0e004b30-24b3-4e79-b777-b0dbfff033f2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:47:02.423093 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:47:02.423093 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:47:02.423093 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:47:02.423093 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:47:02.423093 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:47:02.423093 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:47:02.423093 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:47:02.423093 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:47:02.423093 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:47:02.423093 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:47:02.423093 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:47:02.423093 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 10 11:47:02.423093 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 10 11:47:02.423093 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 10 11:47:02.423093 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 10 11:47:02.424786 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:47:02.424786 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:47:02.424786 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:47:02.424786 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:47:02.424786 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:47:02.424786 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:47:02.424786 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:47:02.424786 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application with Lock( Oct 10 11:47:02.424786 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:47:02.424786 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:47:02.424786 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:47:02.424786 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:47:02.424786 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:47:02.424786 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:47:02.424786 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:47:02.424786 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:47:02.424786 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 10 11:47:02.426354 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 10 11:47:02.426354 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:47:02.426354 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Oct 10 11:47:02.426354 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 325/622] 10.222.0.46 () {56 vars in 1108 bytes} [Mon Oct 10 11:47:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 619 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 10 11:47:02.435094 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e004b30-24b3-4e79-b777-b0dbfff033f2 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:02.490277 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786b5df1-eb14-460d-bb90-0b65a3f0e5ed None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:02.574712 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be618c02-55d6-4931-a085-8af03e0ab620 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:02.574712 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be618c02-55d6-4931-a085-8af03e0ab620 None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:02.574712 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be618c02-55d6-4931-a085-8af03e0ab620 None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:02.574712 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be618c02-55d6-4931-a085-8af03e0ab620 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:02.585682 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be618c02-55d6-4931-a085-8af03e0ab620 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:02.588553 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be618c02-55d6-4931-a085-8af03e0ab620 None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:02.614608 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application [None req-be618c02-55d6-4931-a085-8af03e0ab620 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Oct 10 11:47:02.614608 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:47:02.614608 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:47:02.614608 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:47:02.614608 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:47:02.614608 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:47:02.614608 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:47:02.614608 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:47:02.614608 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:47:02.614608 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:47:02.614608 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:47:02.614608 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:47:02.614608 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 10 11:47:02.614608 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 10 11:47:02.614608 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 10 11:47:02.614608 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 10 11:47:02.616533 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:47:02.616533 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:47:02.616533 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 10 11:47:02.616533 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 10 11:47:02.616533 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 10 11:47:02.616533 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 10 11:47:02.616533 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:47:02.616533 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:47:02.616533 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:47:02.616533 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:47:02.616533 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:47:02.616533 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application with Lock( Oct 10 11:47:02.616533 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:47:02.616533 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:47:02.616533 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:47:02.616533 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:47:02.616533 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:47:02.616533 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:47:02.618367 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:47:02.618367 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:47:02.618367 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 10 11:47:02.618367 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 10 11:47:02.618367 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 10 11:47:02.618367 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 10 11:47:02.618367 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 10 11:47:02.618367 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 10 11:47:02.618367 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Oct 10 11:47:02.618367 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Oct 10 11:47:02.618367 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 298/623] 10.222.0.46 () {56 vars in 1099 bytes} [Mon Oct 10 11:47:02 2022] GET /identity/v3/users/cinder => generated 84 bytes in 500 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 10 11:47:02.627616 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be618c02-55d6-4931-a085-8af03e0ab620 None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:02.778381 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e004b30-24b3-4e79-b777-b0dbfff033f2 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:02.820897 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:02.843142 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:02.843762 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be618c02-55d6-4931-a085-8af03e0ab620 None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:02.872662 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551223b2-b7fb-4733-a0ee-64b20e6aba18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:02.874283 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551223b2-b7fb-4733-a0ee-64b20e6aba18 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:02.874823 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551223b2-b7fb-4733-a0ee-64b20e6aba18 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:02.885124 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551223b2-b7fb-4733-a0ee-64b20e6aba18 None admin] PATH_INFO: `/v3/domains` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:02.885124 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551223b2-b7fb-4733-a0ee-64b20e6aba18 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:47:02.885124 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551223b2-b7fb-4733-a0ee-64b20e6aba18 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:02.885124 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551223b2-b7fb-4733-a0ee-64b20e6aba18 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:02.905009 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36708595-9b3d-46c1-a9fc-a6c9a47b9eda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:02.905792 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36708595-9b3d-46c1-a9fc-a6c9a47b9eda None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:02.906172 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36708595-9b3d-46c1-a9fc-a6c9a47b9eda None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:02.906602 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36708595-9b3d-46c1-a9fc-a6c9a47b9eda None admin] PATH_INFO: `/v3/users` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:02.907492 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36708595-9b3d-46c1-a9fc-a6c9a47b9eda None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:47:02.907898 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36708595-9b3d-46c1-a9fc-a6c9a47b9eda None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:02.908740 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 326/624] 10.222.0.46 () {56 vars in 1109 bytes} [Mon Oct 10 11:47:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 475 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:47:02.910485 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36708595-9b3d-46c1-a9fc-a6c9a47b9eda None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:02.912700 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2538d6-d54e-48f1-9adf-4474c520f486 None None] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:02.913178 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2538d6-d54e-48f1-9adf-4474c520f486 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:02.913658 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2538d6-d54e-48f1-9adf-4474c520f486 None None] PATH_INFO: `/` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:02.917078 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 327/625] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 10 11:47:02 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:47:02.920583 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2538d6-d54e-48f1-9adf-4474c520f486 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:02.937300 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 299/626] 10.222.0.46 () {56 vars in 1137 bytes} [Mon Oct 10 11:47:02 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 11:47:02.938610 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a84197-a702-483a-b0bb-742aaabdfaee None None] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:02.943096 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a84197-a702-483a-b0bb-742aaabdfaee None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:02.943554 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a84197-a702-483a-b0bb-742aaabdfaee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:02.963762 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.auth.core [None req-81a84197-a702-483a-b0bb-742aaabdfaee None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:47:02.964980 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-81a84197-a702-483a-b0bb-742aaabdfaee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:47:03.087564 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 300/627] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 10 11:47:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 10 11:47:03.089282 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a84197-a702-483a-b0bb-742aaabdfaee None None] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:03.089767 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2538d6-d54e-48f1-9adf-4474c520f486 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:03.162014 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75504eda-0165-46d3-9b2b-f4fd96d3e4bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:03.163646 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75504eda-0165-46d3-9b2b-f4fd96d3e4bf None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:03.164398 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75504eda-0165-46d3-9b2b-f4fd96d3e4bf None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:03.165148 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75504eda-0165-46d3-9b2b-f4fd96d3e4bf None admin] PATH_INFO: `/v3/projects/service` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:03.180324 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75504eda-0165-46d3-9b2b-f4fd96d3e4bf None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:03.183364 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75504eda-0165-46d3-9b2b-f4fd96d3e4bf None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:03.219901 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application [None req-75504eda-0165-46d3-9b2b-f4fd96d3e4bf None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 10 11:47:03.219901 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:47:03.219901 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:47:03.219901 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:47:03.219901 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:47:03.219901 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:47:03.219901 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:47:03.219901 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:47:03.219901 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:47:03.219901 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:47:03.219901 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:47:03.219901 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:47:03.219901 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 10 11:47:03.219901 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 10 11:47:03.219901 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 10 11:47:03.221583 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 10 11:47:03.221583 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:47:03.221583 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:47:03.221583 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:47:03.221583 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:47:03.221583 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:47:03.221583 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:47:03.221583 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:47:03.221583 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application with Lock( Oct 10 11:47:03.221583 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:47:03.221583 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:47:03.221583 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:47:03.221583 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:47:03.221583 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:47:03.221583 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:47:03.221583 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:47:03.221583 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:47:03.223307 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 10 11:47:03.223307 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 10 11:47:03.223307 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 10 11:47:03.223307 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 10 11:47:03.223307 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 10 11:47:03.223307 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 10 11:47:03.223307 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 10 11:47:03.223307 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Oct 10 11:47:03.227513 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 328/628] 10.222.0.46 () {56 vars in 1111 bytes} [Mon Oct 10 11:47:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 310 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:47:03.237774 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75504eda-0165-46d3-9b2b-f4fd96d3e4bf None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:03.299390 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:03.341321 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a84197-a702-483a-b0bb-742aaabdfaee None None] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:03.374868 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:03.422334 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32da9a4d-8154-455c-af18-8020e177ec03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:03.424147 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32da9a4d-8154-455c-af18-8020e177ec03 None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:03.425674 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32da9a4d-8154-455c-af18-8020e177ec03 None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:03.426812 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32da9a4d-8154-455c-af18-8020e177ec03 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:03.516306 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75504eda-0165-46d3-9b2b-f4fd96d3e4bf None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:03.528702 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32da9a4d-8154-455c-af18-8020e177ec03 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:03.530378 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32da9a4d-8154-455c-af18-8020e177ec03 None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:03.603271 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442e9385-4396-4ac3-96d7-cdce5f1be095 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:03.605574 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442e9385-4396-4ac3-96d7-cdce5f1be095 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:03.606323 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442e9385-4396-4ac3-96d7-cdce5f1be095 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:03.607082 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442e9385-4396-4ac3-96d7-cdce5f1be095 None admin] PATH_INFO: `/v3/projects` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:03.609518 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442e9385-4396-4ac3-96d7-cdce5f1be095 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:47:03.610704 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442e9385-4396-4ac3-96d7-cdce5f1be095 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:03.614084 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442e9385-4396-4ac3-96d7-cdce5f1be095 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:03.647510 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 329/629] 10.222.0.46 () {56 vars in 1148 bytes} [Mon Oct 10 11:47:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 410 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:47:03.652193 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442e9385-4396-4ac3-96d7-cdce5f1be095 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:03.665182 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:03.703158 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 301/630] 10.222.0.46 () {58 vars in 1282 bytes} [Mon Oct 10 11:47:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 614 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 10 11:47:03.704167 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32da9a4d-8154-455c-af18-8020e177ec03 None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:03.799235 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442e9385-4396-4ac3-96d7-cdce5f1be095 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:03.808112 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:03.827800 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32da9a4d-8154-455c-af18-8020e177ec03 None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:03.839147 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17149e9-49db-4707-ace7-264ae23a9053 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:03.840394 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17149e9-49db-4707-ace7-264ae23a9053 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:03.841549 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17149e9-49db-4707-ace7-264ae23a9053 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:03.841968 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17149e9-49db-4707-ace7-264ae23a9053 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:03.857578 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17149e9-49db-4707-ace7-264ae23a9053 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 10 11:47:03.867405 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application [None req-d17149e9-49db-4707-ace7-264ae23a9053 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 10 11:47:03.867405 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:47:03.867405 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:47:03.867405 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:47:03.867405 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:47:03.867405 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:47:03.867405 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:47:03.867405 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:47:03.867405 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:47:03.867405 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:47:03.867405 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:47:03.867405 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:47:03.867405 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 10 11:47:03.867405 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 10 11:47:03.867405 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 10 11:47:03.867405 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 10 11:47:03.869487 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 10 11:47:03.869487 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 10 11:47:03.869487 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 10 11:47:03.869487 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 10 11:47:03.869487 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:47:03.869487 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:47:03.869487 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:47:03.869487 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:47:03.869487 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:47:03.869487 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:47:03.869487 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:47:03.869487 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application with Lock( Oct 10 11:47:03.869487 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:47:03.869487 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:47:03.869487 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:47:03.869487 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:47:03.871420 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:47:03.871420 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:47:03.871420 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:47:03.871420 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:47:03.871420 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 10 11:47:03.871420 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 10 11:47:03.871420 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 10 11:47:03.871420 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 10 11:47:03.871420 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 10 11:47:03.871420 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 10 11:47:03.871420 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 10 11:47:03.871420 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Oct 10 11:47:03.871420 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 330/631] 10.222.0.46 () {56 vars in 1102 bytes} [Mon Oct 10 11:47:03 2022] GET /identity/v3/roles/service => generated 85 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:47:03.873355 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabbdc2f-26a7-4953-9ba3-a330c7d736a8 None None] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:03.873757 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabbdc2f-26a7-4953-9ba3-a330c7d736a8 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:03.874228 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabbdc2f-26a7-4953-9ba3-a330c7d736a8 None None] PATH_INFO: `/` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:03.876699 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 331/632] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 10 11:47:03 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:47:03.878577 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabbdc2f-26a7-4953-9ba3-a330c7d736a8 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:03.889715 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01448c95-a5cf-4f9b-b7b4-3836c7a57c15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:03.891250 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01448c95-a5cf-4f9b-b7b4-3836c7a57c15 None admin] REQUEST_METHOD: `PUT` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:03.891941 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01448c95-a5cf-4f9b-b7b4-3836c7a57c15 None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:03.892605 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01448c95-a5cf-4f9b-b7b4-3836c7a57c15 None admin] PATH_INFO: `/v3/projects/4ad3772f7b6a47c28b7ba37a9316a3c1/users/15ef3d53ea484237adb0b3d64fdc5480/roles/3c35be38787740f4834c728a2ce73b6b` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:03.920331 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01448c95-a5cf-4f9b-b7b4-3836c7a57c15 None admin] RBAC: Authorizing `identity:create_grant(project_id=4ad3772f7b6a47c28b7ba37a9316a3c1, user_id=15ef3d53ea484237adb0b3d64fdc5480, role_id=3c35be38787740f4834c728a2ce73b6b)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:03.923256 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01448c95-a5cf-4f9b-b7b4-3836c7a57c15 None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:04.021690 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 302/633] 10.222.0.46 () {58 vars in 1439 bytes} [Mon Oct 10 11:47:03 2022] PUT /identity/v3/projects/4ad3772f7b6a47c28b7ba37a9316a3c1/users/15ef3d53ea484237adb0b3d64fdc5480/roles/3c35be38787740f4834c728a2ce73b6b => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 11:47:04.022450 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01448c95-a5cf-4f9b-b7b4-3836c7a57c15 None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:04.043857 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabbdc2f-26a7-4953-9ba3-a330c7d736a8 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:04.083658 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:04.112829 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601a263c-dfbc-4b6b-9b12-a704644a094b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:04.113999 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601a263c-dfbc-4b6b-9b12-a704644a094b None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:04.114465 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601a263c-dfbc-4b6b-9b12-a704644a094b None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:04.114922 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601a263c-dfbc-4b6b-9b12-a704644a094b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:04.131539 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601a263c-dfbc-4b6b-9b12-a704644a094b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:04.133590 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601a263c-dfbc-4b6b-9b12-a704644a094b None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:04.156753 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application [None req-601a263c-dfbc-4b6b-9b12-a704644a094b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:47:04.156753 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:47:04.156753 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:47:04.156753 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:47:04.156753 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:47:04.156753 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:47:04.156753 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:47:04.156753 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:47:04.156753 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:47:04.156753 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:47:04.156753 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:47:04.156753 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:47:04.156753 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 10 11:47:04.156753 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 10 11:47:04.156753 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 10 11:47:04.156753 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 10 11:47:04.157944 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:47:04.157944 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:47:04.157944 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:47:04.157944 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:47:04.157944 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:47:04.157944 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:47:04.157944 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:47:04.157944 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application with Lock( Oct 10 11:47:04.157944 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:47:04.157944 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:47:04.157944 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:47:04.157944 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:47:04.157944 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:47:04.157944 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:47:04.157944 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:47:04.157944 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:47:04.157944 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 10 11:47:04.159063 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 10 11:47:04.159063 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:47:04.159063 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Oct 10 11:47:04.161330 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 332/634] 10.222.0.46 () {56 vars in 1108 bytes} [Mon Oct 10 11:47:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:47:04.163075 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c103f78c-361e-42d7-b518-5259beaebbf9 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:04.163563 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c103f78c-361e-42d7-b518-5259beaebbf9 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:04.163986 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c103f78c-361e-42d7-b518-5259beaebbf9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:04.193324 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-c103f78c-361e-42d7-b518-5259beaebbf9 None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:47:04.195945 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-c103f78c-361e-42d7-b518-5259beaebbf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:47:04.235119 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01448c95-a5cf-4f9b-b7b4-3836c7a57c15 None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:04.235339 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:04.293096 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a04295-7a10-4fba-8a0f-95c2d2ae1eea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:04.293768 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a04295-7a10-4fba-8a0f-95c2d2ae1eea None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:04.294125 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a04295-7a10-4fba-8a0f-95c2d2ae1eea None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:04.294474 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a04295-7a10-4fba-8a0f-95c2d2ae1eea None admin] PATH_INFO: `/v3/roles` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:04.295974 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a04295-7a10-4fba-8a0f-95c2d2ae1eea None admin] RBAC: Adding query filter params (name=service) {{(pid=81676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:47:04.296371 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a04295-7a10-4fba-8a0f-95c2d2ae1eea None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:04.298034 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a04295-7a10-4fba-8a0f-95c2d2ae1eea None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:04.303225 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 333/635] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 10 11:47:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 10 11:47:04.304409 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c103f78c-361e-42d7-b518-5259beaebbf9 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:04.314636 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 303/636] 10.222.0.46 () {56 vars in 1103 bytes} [Mon Oct 10 11:47:04 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:47:04.321155 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a04295-7a10-4fba-8a0f-95c2d2ae1eea None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:04.439082 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:04.446595 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c103f78c-361e-42d7-b518-5259beaebbf9 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:04.449360 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a04295-7a10-4fba-8a0f-95c2d2ae1eea None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:04.497806 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9102f50-2e6b-40ed-afd2-3ed6bb88e02d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:04.499357 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9102f50-2e6b-40ed-afd2-3ed6bb88e02d None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:04.499844 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9102f50-2e6b-40ed-afd2-3ed6bb88e02d None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:04.500389 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9102f50-2e6b-40ed-afd2-3ed6bb88e02d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:04.506516 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cafa39a-5df3-4c61-8692-0b39f8f8ff50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:04.507197 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cafa39a-5df3-4c61-8692-0b39f8f8ff50 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:04.507484 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cafa39a-5df3-4c61-8692-0b39f8f8ff50 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:04.507766 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cafa39a-5df3-4c61-8692-0b39f8f8ff50 None admin] PATH_INFO: `/v3/domains` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:04.508431 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cafa39a-5df3-4c61-8692-0b39f8f8ff50 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:47:04.508746 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cafa39a-5df3-4c61-8692-0b39f8f8ff50 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:04.510931 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cafa39a-5df3-4c61-8692-0b39f8f8ff50 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:04.514380 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9102f50-2e6b-40ed-afd2-3ed6bb88e02d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:04.516202 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9102f50-2e6b-40ed-afd2-3ed6bb88e02d None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:04.527229 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 334/637] 10.222.0.46 () {56 vars in 1109 bytes} [Mon Oct 10 11:47:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:47:04.528585 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce26f9c-ec45-43af-b1d6-8f8baa9bfa1f None None] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:04.528880 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce26f9c-ec45-43af-b1d6-8f8baa9bfa1f None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:04.529158 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce26f9c-ec45-43af-b1d6-8f8baa9bfa1f None None] PATH_INFO: `/` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:04.535778 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 335/638] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 10 11:47:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:47:04.536883 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce26f9c-ec45-43af-b1d6-8f8baa9bfa1f None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:04.541918 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application [None req-c9102f50-2e6b-40ed-afd2-3ed6bb88e02d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:47:04.541918 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:47:04.541918 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:47:04.541918 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:47:04.541918 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:47:04.541918 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:47:04.541918 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:47:04.541918 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:47:04.541918 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:47:04.541918 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:47:04.541918 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:47:04.541918 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:47:04.541918 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 10 11:47:04.541918 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 10 11:47:04.541918 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 10 11:47:04.541918 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 10 11:47:04.542961 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:47:04.542961 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:47:04.542961 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:47:04.542961 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:47:04.542961 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:47:04.542961 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:47:04.542961 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:47:04.542961 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application with Lock( Oct 10 11:47:04.542961 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:47:04.542961 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:47:04.542961 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:47:04.542961 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:47:04.542961 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:47:04.542961 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:47:04.542961 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:47:04.542961 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:47:04.542961 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 10 11:47:04.543889 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 10 11:47:04.543889 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:47:04.543889 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Oct 10 11:47:04.546832 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 304/639] 10.222.0.46 () {56 vars in 1108 bytes} [Mon Oct 10 11:47:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:47:04.548449 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcc19ba-f850-4940-a96e-7ecfad0787e0 None None] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:04.548953 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcc19ba-f850-4940-a96e-7ecfad0787e0 None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:04.549473 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcc19ba-f850-4940-a96e-7ecfad0787e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:04.551212 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:04.570438 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.auth.core [None req-5dcc19ba-f850-4940-a96e-7ecfad0787e0 None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:47:04.572153 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-5dcc19ba-f850-4940-a96e-7ecfad0787e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:47:04.648584 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 305/640] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 10 11:47:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 10 11:47:04.650099 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcc19ba-f850-4940-a96e-7ecfad0787e0 None None] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:04.667329 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce26f9c-ec45-43af-b1d6-8f8baa9bfa1f None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:04.709594 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8c26dd-0981-4af5-b340-1eb869c33364 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:04.709945 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8c26dd-0981-4af5-b340-1eb869c33364 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:04.710143 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8c26dd-0981-4af5-b340-1eb869c33364 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:04.710423 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8c26dd-0981-4af5-b340-1eb869c33364 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:04.718659 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8c26dd-0981-4af5-b340-1eb869c33364 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:04.720223 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8c26dd-0981-4af5-b340-1eb869c33364 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:04.733646 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application [None req-9d8c26dd-0981-4af5-b340-1eb869c33364 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 10 11:47:04.733646 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:47:04.733646 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:47:04.733646 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:47:04.733646 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:47:04.733646 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:47:04.733646 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:47:04.733646 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:47:04.733646 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:47:04.733646 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:47:04.733646 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:47:04.733646 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:47:04.733646 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 10 11:47:04.733646 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 10 11:47:04.733646 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 10 11:47:04.734592 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 10 11:47:04.734592 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:47:04.734592 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:47:04.734592 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:47:04.734592 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:47:04.734592 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:47:04.734592 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:47:04.734592 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:47:04.734592 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application with Lock( Oct 10 11:47:04.734592 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:47:04.734592 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:47:04.734592 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:47:04.734592 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:47:04.734592 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:47:04.734592 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:47:04.734592 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:47:04.734592 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:47:04.735509 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 10 11:47:04.735509 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 10 11:47:04.735509 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 10 11:47:04.735509 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 10 11:47:04.735509 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 10 11:47:04.735509 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 10 11:47:04.735509 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 10 11:47:04.735509 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Oct 10 11:47:04.735509 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 336/641] 10.222.0.46 () {56 vars in 1111 bytes} [Mon Oct 10 11:47:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:47:04.735982 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8c26dd-0981-4af5-b340-1eb869c33364 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:04.770058 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:04.796270 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcc19ba-f850-4940-a96e-7ecfad0787e0 None None] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:04.830947 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:04.848061 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc863392-8117-4f26-a5fb-dee3fa2bb69e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:04.849173 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc863392-8117-4f26-a5fb-dee3fa2bb69e None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:04.850013 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc863392-8117-4f26-a5fb-dee3fa2bb69e None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:04.850610 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc863392-8117-4f26-a5fb-dee3fa2bb69e None admin] PATH_INFO: `/v3/domains` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:04.851628 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc863392-8117-4f26-a5fb-dee3fa2bb69e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:47:04.852295 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc863392-8117-4f26-a5fb-dee3fa2bb69e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:04.854198 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc863392-8117-4f26-a5fb-dee3fa2bb69e None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:04.871906 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8c26dd-0981-4af5-b340-1eb869c33364 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:04.872440 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 306/642] 10.222.0.46 () {56 vars in 1109 bytes} [Mon Oct 10 11:47:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:47:04.874046 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc863392-8117-4f26-a5fb-dee3fa2bb69e None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:04.912853 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d408533-03dd-4894-a016-57822d660cda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:04.913619 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d408533-03dd-4894-a016-57822d660cda None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:04.913955 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d408533-03dd-4894-a016-57822d660cda None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:04.914305 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d408533-03dd-4894-a016-57822d660cda None admin] PATH_INFO: `/v3/roles/service` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:04.929197 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d408533-03dd-4894-a016-57822d660cda None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 10 11:47:04.936788 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application [None req-6d408533-03dd-4894-a016-57822d660cda None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 10 11:47:04.936788 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:47:04.936788 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:47:04.936788 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:47:04.936788 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:47:04.936788 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:47:04.936788 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:47:04.936788 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:47:04.936788 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:47:04.936788 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:47:04.936788 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:47:04.936788 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:47:04.936788 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 10 11:47:04.936788 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 10 11:47:04.936788 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 10 11:47:04.936788 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 10 11:47:04.937828 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 10 11:47:04.937828 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 10 11:47:04.937828 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 10 11:47:04.937828 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 10 11:47:04.937828 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:47:04.937828 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:47:04.937828 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:47:04.937828 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:47:04.937828 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:47:04.937828 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:47:04.937828 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:47:04.937828 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application with Lock( Oct 10 11:47:04.937828 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:47:04.937828 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:47:04.937828 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:47:04.937828 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:47:04.938662 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:47:04.938662 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:47:04.938662 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:47:04.938662 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:47:04.938662 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 10 11:47:04.938662 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 10 11:47:04.938662 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 10 11:47:04.938662 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 10 11:47:04.938662 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 10 11:47:04.938662 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 10 11:47:04.938662 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 10 11:47:04.938662 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Oct 10 11:47:04.938662 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 337/643] 10.222.0.46 () {56 vars in 1102 bytes} [Mon Oct 10 11:47:04 2022] GET /identity/v3/roles/service => generated 85 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:47:04.939405 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d408533-03dd-4894-a016-57822d660cda None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:05.009122 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc863392-8117-4f26-a5fb-dee3fa2bb69e None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:05.046200 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa69c1b8-0e0f-4f8e-8221-0281ef71884d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:05.047034 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d408533-03dd-4894-a016-57822d660cda None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:05.047451 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa69c1b8-0e0f-4f8e-8221-0281ef71884d None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:05.047910 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa69c1b8-0e0f-4f8e-8221-0281ef71884d None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:05.048428 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa69c1b8-0e0f-4f8e-8221-0281ef71884d None admin] PATH_INFO: `/v3/projects` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:05.049247 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa69c1b8-0e0f-4f8e-8221-0281ef71884d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:47:05.049568 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa69c1b8-0e0f-4f8e-8221-0281ef71884d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:05.051046 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa69c1b8-0e0f-4f8e-8221-0281ef71884d None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:05.054207 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:05.064646 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 307/644] 10.222.0.46 () {56 vars in 1148 bytes} [Mon Oct 10 11:47:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:47:05.072378 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa69c1b8-0e0f-4f8e-8221-0281ef71884d None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:05.081977 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:05.092678 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04cce171-84d4-4010-aba6-75bfc9518ceb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:05.093345 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cce171-84d4-4010-aba6-75bfc9518ceb None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:05.093618 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cce171-84d4-4010-aba6-75bfc9518ceb None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:05.101807 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cce171-84d4-4010-aba6-75bfc9518ceb None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:05.102829 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04cce171-84d4-4010-aba6-75bfc9518ceb None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:05.104309 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04cce171-84d4-4010-aba6-75bfc9518ceb None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:05.118025 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application [None req-04cce171-84d4-4010-aba6-75bfc9518ceb None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Oct 10 11:47:05.118025 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:47:05.118025 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:47:05.118025 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:47:05.118025 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:47:05.118025 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:47:05.118025 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:47:05.118025 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:47:05.118025 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:47:05.118025 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:47:05.118025 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:47:05.118025 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:47:05.118025 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 10 11:47:05.118025 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 10 11:47:05.118025 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 10 11:47:05.118025 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 10 11:47:05.119119 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:47:05.119119 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:47:05.119119 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 10 11:47:05.119119 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 10 11:47:05.119119 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 10 11:47:05.119119 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 10 11:47:05.119119 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:47:05.119119 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:47:05.119119 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:47:05.119119 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:47:05.119119 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:47:05.119119 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application with Lock( Oct 10 11:47:05.119119 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:47:05.119119 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:47:05.119119 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:47:05.119119 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:47:05.119119 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:47:05.119119 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:47:05.120181 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:47:05.120181 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:47:05.120181 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 10 11:47:05.120181 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 10 11:47:05.120181 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 10 11:47:05.120181 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 10 11:47:05.120181 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 10 11:47:05.120181 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 10 11:47:05.120181 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Oct 10 11:47:05.120181 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Oct 10 11:47:05.120181 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 338/645] 10.222.0.46 () {56 vars in 1102 bytes} [Mon Oct 10 11:47:04 2022] GET /identity/v3/users/neutron => generated 85 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:47:05.120181 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04cce171-84d4-4010-aba6-75bfc9518ceb None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:05.183023 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa69c1b8-0e0f-4f8e-8221-0281ef71884d None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:05.221015 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684b8328-0cff-400b-96b5-3604c6cbadac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:05.221665 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684b8328-0cff-400b-96b5-3604c6cbadac None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:05.221953 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684b8328-0cff-400b-96b5-3604c6cbadac None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:05.222261 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684b8328-0cff-400b-96b5-3604c6cbadac None admin] PATH_INFO: `/v3/roles/service` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:05.234422 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684b8328-0cff-400b-96b5-3604c6cbadac None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 10 11:47:05.239056 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04cce171-84d4-4010-aba6-75bfc9518ceb None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:05.239509 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application [None req-684b8328-0cff-400b-96b5-3604c6cbadac None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 10 11:47:05.239509 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:47:05.239509 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:47:05.239509 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:47:05.239509 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:47:05.239509 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:47:05.239509 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:47:05.239509 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:47:05.239509 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:47:05.239509 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:47:05.239509 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:47:05.239509 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:47:05.239509 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 10 11:47:05.239509 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 10 11:47:05.239509 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 10 11:47:05.239509 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 10 11:47:05.240573 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 10 11:47:05.240573 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 10 11:47:05.240573 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 10 11:47:05.240573 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 10 11:47:05.240573 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:47:05.240573 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:47:05.240573 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:47:05.240573 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:47:05.240573 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:47:05.240573 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:47:05.240573 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:47:05.240573 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application with Lock( Oct 10 11:47:05.240573 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:47:05.240573 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:47:05.240573 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:47:05.240573 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:47:05.241439 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:47:05.241439 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:47:05.241439 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:47:05.241439 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:47:05.241439 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 10 11:47:05.241439 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 10 11:47:05.241439 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 10 11:47:05.241439 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 10 11:47:05.241439 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 10 11:47:05.241439 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 10 11:47:05.241439 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 10 11:47:05.241439 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Oct 10 11:47:05.241439 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 308/646] 10.222.0.46 () {56 vars in 1102 bytes} [Mon Oct 10 11:47:05 2022] GET /identity/v3/roles/service => generated 85 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:47:05.242108 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684b8328-0cff-400b-96b5-3604c6cbadac None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:05.284843 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee03bc98-f27b-44af-bcca-b8fedbdc0a75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:05.285472 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee03bc98-f27b-44af-bcca-b8fedbdc0a75 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:05.285754 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee03bc98-f27b-44af-bcca-b8fedbdc0a75 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:05.286030 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee03bc98-f27b-44af-bcca-b8fedbdc0a75 None admin] PATH_INFO: `/v3/roles` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:05.286740 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee03bc98-f27b-44af-bcca-b8fedbdc0a75 None admin] RBAC: Adding query filter params (name=service) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:47:05.287054 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee03bc98-f27b-44af-bcca-b8fedbdc0a75 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:05.288436 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee03bc98-f27b-44af-bcca-b8fedbdc0a75 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:05.301822 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 339/647] 10.222.0.46 () {56 vars in 1103 bytes} [Mon Oct 10 11:47:05 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:47:05.301822 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee03bc98-f27b-44af-bcca-b8fedbdc0a75 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:05.318418 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:05.357711 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:05.388167 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684b8328-0cff-400b-96b5-3604c6cbadac None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:05.428018 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8528aab5-a44f-46c3-875d-ba92079c122a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:05.428792 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8528aab5-a44f-46c3-875d-ba92079c122a None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:05.429112 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8528aab5-a44f-46c3-875d-ba92079c122a None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:05.429498 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8528aab5-a44f-46c3-875d-ba92079c122a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:05.430141 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8528aab5-a44f-46c3-875d-ba92079c122a None admin] RBAC: Adding query filter params (role.id=3c35be38787740f4834c728a2ce73b6b, scope.project.id=4ad3772f7b6a47c28b7ba37a9316a3c1, user.id=d3ae093470484b93b23dcfae1657df9e) {{(pid=81676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:47:05.430449 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8528aab5-a44f-46c3-875d-ba92079c122a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:05.431869 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee03bc98-f27b-44af-bcca-b8fedbdc0a75 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:05.432062 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8528aab5-a44f-46c3-875d-ba92079c122a None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:05.446713 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 309/648] 10.222.0.46 () {56 vars in 1374 bytes} [Mon Oct 10 11:47:05 2022] GET /identity/v3/role_assignments?user.id=d3ae093470484b93b23dcfae1657df9e&scope.project.id=4ad3772f7b6a47c28b7ba37a9316a3c1&role.id=3c35be38787740f4834c728a2ce73b6b => generated 608 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:47:05.446713 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8528aab5-a44f-46c3-875d-ba92079c122a None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:05.504821 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db48b22a-ce2a-4d3c-b643-0642897f4409 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:05.506623 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db48b22a-ce2a-4d3c-b643-0642897f4409 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:05.507225 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db48b22a-ce2a-4d3c-b643-0642897f4409 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:05.507585 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db48b22a-ce2a-4d3c-b643-0642897f4409 None admin] PATH_INFO: `/v3/users` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:05.508543 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db48b22a-ce2a-4d3c-b643-0642897f4409 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:47:05.508944 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db48b22a-ce2a-4d3c-b643-0642897f4409 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:05.510630 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db48b22a-ce2a-4d3c-b643-0642897f4409 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:05.546756 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 340/649] 10.222.0.46 () {56 vars in 1139 bytes} [Mon Oct 10 11:47:05 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:47:05.546756 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db48b22a-ce2a-4d3c-b643-0642897f4409 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:05.604742 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8528aab5-a44f-46c3-875d-ba92079c122a None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:05.640044 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:05.671901 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e020671f-5a5a-4bf5-b91e-23ffd8afd522 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:05.675059 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e020671f-5a5a-4bf5-b91e-23ffd8afd522 None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:05.675984 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e020671f-5a5a-4bf5-b91e-23ffd8afd522 None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:05.677088 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e020671f-5a5a-4bf5-b91e-23ffd8afd522 None admin] PATH_INFO: `/v3/roles` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:05.680145 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e020671f-5a5a-4bf5-b91e-23ffd8afd522 None admin] RBAC: Adding query filter params (name=service) {{(pid=81676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:47:05.681533 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e020671f-5a5a-4bf5-b91e-23ffd8afd522 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:05.687027 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e020671f-5a5a-4bf5-b91e-23ffd8afd522 None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:05.689782 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db48b22a-ce2a-4d3c-b643-0642897f4409 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:05.704205 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:05.714729 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 310/650] 10.222.0.46 () {56 vars in 1103 bytes} [Mon Oct 10 11:47:05 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:47:05.714729 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e020671f-5a5a-4bf5-b91e-23ffd8afd522 None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:05.758277 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c9135a-baee-4b1f-b905-cc1dcaa2ab58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:05.759153 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c9135a-baee-4b1f-b905-cc1dcaa2ab58 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:05.759434 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c9135a-baee-4b1f-b905-cc1dcaa2ab58 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:05.759710 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c9135a-baee-4b1f-b905-cc1dcaa2ab58 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:05.777588 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c9135a-baee-4b1f-b905-cc1dcaa2ab58 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:05.779785 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c9135a-baee-4b1f-b905-cc1dcaa2ab58 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:05.804873 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application [None req-b1c9135a-baee-4b1f-b905-cc1dcaa2ab58 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:47:05.804873 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:47:05.804873 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:47:05.804873 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:47:05.804873 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:47:05.804873 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:47:05.804873 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:47:05.804873 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:47:05.804873 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:47:05.804873 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:47:05.804873 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:47:05.804873 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:47:05.804873 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 10 11:47:05.804873 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 10 11:47:05.804873 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 10 11:47:05.804873 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 10 11:47:05.805893 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:47:05.805893 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:47:05.805893 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:47:05.805893 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:47:05.805893 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:47:05.805893 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:47:05.805893 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:47:05.805893 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application with Lock( Oct 10 11:47:05.805893 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:47:05.805893 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:47:05.805893 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:47:05.805893 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:47:05.805893 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:47:05.805893 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:47:05.805893 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:47:05.805893 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:47:05.805893 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 10 11:47:05.806839 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 10 11:47:05.806839 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:47:05.806839 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Oct 10 11:47:05.806839 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 341/651] 10.222.0.46 () {56 vars in 1108 bytes} [Mon Oct 10 11:47:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:47:05.806839 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c9135a-baee-4b1f-b905-cc1dcaa2ab58 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:05.886713 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e020671f-5a5a-4bf5-b91e-23ffd8afd522 None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:05.926438 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29d1f1b-5818-490a-8a32-4ae940f4a742 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:05.927667 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29d1f1b-5818-490a-8a32-4ae940f4a742 None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:05.928502 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29d1f1b-5818-490a-8a32-4ae940f4a742 None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:05.929243 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29d1f1b-5818-490a-8a32-4ae940f4a742 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:05.941536 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c9135a-baee-4b1f-b905-cc1dcaa2ab58 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:05.986938 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:05.989001 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33271204-a1e3-45a5-9087-8996a6740990 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:05.989658 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33271204-a1e3-45a5-9087-8996a6740990 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:05.989941 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33271204-a1e3-45a5-9087-8996a6740990 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:05.990222 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33271204-a1e3-45a5-9087-8996a6740990 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:06.004803 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33271204-a1e3-45a5-9087-8996a6740990 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:06.006280 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33271204-a1e3-45a5-9087-8996a6740990 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:06.012123 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29d1f1b-5818-490a-8a32-4ae940f4a742 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:06.014592 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29d1f1b-5818-490a-8a32-4ae940f4a742 None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:06.024206 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application [None req-33271204-a1e3-45a5-9087-8996a6740990 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:47:06.024206 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:47:06.024206 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:47:06.024206 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:47:06.024206 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:47:06.024206 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:47:06.024206 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:47:06.024206 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:47:06.024206 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:47:06.024206 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:47:06.024206 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:47:06.024206 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:47:06.024206 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 10 11:47:06.024206 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 10 11:47:06.024206 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 10 11:47:06.024206 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 10 11:47:06.025219 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:47:06.025219 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:47:06.025219 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:47:06.025219 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:47:06.025219 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:47:06.025219 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:47:06.025219 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:47:06.025219 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application with Lock( Oct 10 11:47:06.025219 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:47:06.025219 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:47:06.025219 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:47:06.025219 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:47:06.025219 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:47:06.025219 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:47:06.025219 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:47:06.025219 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:47:06.025219 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 10 11:47:06.026095 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 10 11:47:06.026095 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:47:06.026095 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Oct 10 11:47:06.026095 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 342/652] 10.222.0.46 () {56 vars in 1108 bytes} [Mon Oct 10 11:47:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:47:06.026316 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33271204-a1e3-45a5-9087-8996a6740990 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:06.046138 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:06.149233 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 311/653] 10.222.0.46 () {58 vars in 1282 bytes} [Mon Oct 10 11:47:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 440 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 10 11:47:06.150855 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29d1f1b-5818-490a-8a32-4ae940f4a742 None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:06.161176 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33271204-a1e3-45a5-9087-8996a6740990 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:06.209409 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc9a693-d654-4e37-89d3-ccdcd0f76f85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:06.210473 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc9a693-d654-4e37-89d3-ccdcd0f76f85 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:06.211012 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc9a693-d654-4e37-89d3-ccdcd0f76f85 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:06.211490 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc9a693-d654-4e37-89d3-ccdcd0f76f85 None admin] PATH_INFO: `/v3/domains` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:06.212617 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc9a693-d654-4e37-89d3-ccdcd0f76f85 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:47:06.213134 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc9a693-d654-4e37-89d3-ccdcd0f76f85 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:06.215968 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc9a693-d654-4e37-89d3-ccdcd0f76f85 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:06.241895 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 343/654] 10.222.0.46 () {56 vars in 1109 bytes} [Mon Oct 10 11:47:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:47:06.243602 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc9a693-d654-4e37-89d3-ccdcd0f76f85 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:06.271824 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:06.292534 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29d1f1b-5818-490a-8a32-4ae940f4a742 None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:06.342210 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89694f8c-9840-40ec-9a02-f19778942a2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:06.342900 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89694f8c-9840-40ec-9a02-f19778942a2a None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:06.343151 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89694f8c-9840-40ec-9a02-f19778942a2a None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:06.343590 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89694f8c-9840-40ec-9a02-f19778942a2a None admin] PATH_INFO: `/v3/domains` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:06.344259 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89694f8c-9840-40ec-9a02-f19778942a2a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:47:06.344619 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89694f8c-9840-40ec-9a02-f19778942a2a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:06.346155 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89694f8c-9840-40ec-9a02-f19778942a2a None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:06.360664 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 312/655] 10.222.0.46 () {56 vars in 1109 bytes} [Mon Oct 10 11:47:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:47:06.361965 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89694f8c-9840-40ec-9a02-f19778942a2a None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:06.386119 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:06.446224 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc9a693-d654-4e37-89d3-ccdcd0f76f85 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:06.510931 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89694f8c-9840-40ec-9a02-f19778942a2a None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:06.512193 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bdc78d-8879-45aa-9f50-1906591d1d79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:06.513076 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bdc78d-8879-45aa-9f50-1906591d1d79 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:06.514068 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bdc78d-8879-45aa-9f50-1906591d1d79 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:06.514564 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bdc78d-8879-45aa-9f50-1906591d1d79 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:06.536082 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73bdc78d-8879-45aa-9f50-1906591d1d79 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:06.538642 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73bdc78d-8879-45aa-9f50-1906591d1d79 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:06.552435 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79caf26d-7b37-4dd4-9534-6b8a05d8a699 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:06.553442 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79caf26d-7b37-4dd4-9534-6b8a05d8a699 None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:06.553931 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79caf26d-7b37-4dd4-9534-6b8a05d8a699 None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:06.554437 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79caf26d-7b37-4dd4-9534-6b8a05d8a699 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:06.563778 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79caf26d-7b37-4dd4-9534-6b8a05d8a699 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:06.565591 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79caf26d-7b37-4dd4-9534-6b8a05d8a699 None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:06.567771 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application [None req-73bdc78d-8879-45aa-9f50-1906591d1d79 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:47:06.567771 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:47:06.567771 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:47:06.567771 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:47:06.567771 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:47:06.567771 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:47:06.567771 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:47:06.567771 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:47:06.567771 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:47:06.567771 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:47:06.567771 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:47:06.567771 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:47:06.567771 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 10 11:47:06.567771 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 10 11:47:06.567771 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 10 11:47:06.567771 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 10 11:47:06.568709 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:47:06.568709 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:47:06.568709 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:47:06.568709 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:47:06.568709 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:47:06.568709 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:47:06.568709 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:47:06.568709 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application with Lock( Oct 10 11:47:06.568709 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:47:06.568709 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:47:06.568709 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:47:06.568709 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:47:06.568709 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:47:06.568709 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:47:06.568709 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:47:06.568709 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:47:06.568709 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 10 11:47:06.569605 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 10 11:47:06.569605 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:47:06.569605 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Oct 10 11:47:06.570191 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 344/656] 10.222.0.46 () {56 vars in 1108 bytes} [Mon Oct 10 11:47:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 327 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:47:06.571786 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bdc78d-8879-45aa-9f50-1906591d1d79 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:06.581321 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application [None req-79caf26d-7b37-4dd4-9534-6b8a05d8a699 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 10 11:47:06.581321 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:47:06.581321 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:47:06.581321 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:47:06.581321 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:47:06.581321 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:47:06.581321 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:47:06.581321 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:47:06.581321 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:47:06.581321 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:47:06.581321 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:47:06.581321 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:47:06.581321 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 10 11:47:06.581321 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 10 11:47:06.581321 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 10 11:47:06.581321 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 10 11:47:06.582298 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:47:06.582298 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:47:06.582298 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 10 11:47:06.582298 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 10 11:47:06.582298 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 10 11:47:06.582298 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 10 11:47:06.582298 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:47:06.582298 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:47:06.582298 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:47:06.582298 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:47:06.582298 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:47:06.582298 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application with Lock( Oct 10 11:47:06.582298 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:47:06.582298 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:47:06.582298 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:47:06.582298 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:47:06.582298 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:47:06.582298 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:47:06.583272 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:47:06.583272 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:47:06.583272 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 10 11:47:06.583272 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 10 11:47:06.583272 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 10 11:47:06.583272 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 10 11:47:06.583272 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 10 11:47:06.583272 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 10 11:47:06.583272 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 10 11:47:06.583272 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Oct 10 11:47:06.585523 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 313/657] 10.222.0.46 () {56 vars in 1093 bytes} [Mon Oct 10 11:47:06 2022] GET /identity/v3/users/nova => generated 82 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:47:06.586560 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79caf26d-7b37-4dd4-9534-6b8a05d8a699 None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:06.609176 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:06.704005 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79caf26d-7b37-4dd4-9534-6b8a05d8a699 None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:06.745577 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b22e19-e30d-4c1e-9bec-cff80227f177 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:06.746594 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b22e19-e30d-4c1e-9bec-cff80227f177 None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:06.747191 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b22e19-e30d-4c1e-9bec-cff80227f177 None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:06.748231 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b22e19-e30d-4c1e-9bec-cff80227f177 None admin] PATH_INFO: `/v3/domains` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:06.749226 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b22e19-e30d-4c1e-9bec-cff80227f177 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:47:06.749762 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b22e19-e30d-4c1e-9bec-cff80227f177 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:06.751424 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b22e19-e30d-4c1e-9bec-cff80227f177 None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:06.765294 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 314/658] 10.222.0.46 () {56 vars in 1109 bytes} [Mon Oct 10 11:47:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:47:06.765796 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bdc78d-8879-45aa-9f50-1906591d1d79 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:06.766761 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b22e19-e30d-4c1e-9bec-cff80227f177 None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:06.812465 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:06.835551 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901ce212-a098-4b6e-9f44-9b8ca3e0979f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:06.837154 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901ce212-a098-4b6e-9f44-9b8ca3e0979f None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:06.838218 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901ce212-a098-4b6e-9f44-9b8ca3e0979f None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:06.838769 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901ce212-a098-4b6e-9f44-9b8ca3e0979f None admin] PATH_INFO: `/v3/users/glance` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:06.849933 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901ce212-a098-4b6e-9f44-9b8ca3e0979f None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:06.852661 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901ce212-a098-4b6e-9f44-9b8ca3e0979f None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:06.863561 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:06.875695 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application [None req-901ce212-a098-4b6e-9f44-9b8ca3e0979f None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Oct 10 11:47:06.875695 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:47:06.875695 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:47:06.875695 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:47:06.875695 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:47:06.875695 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:47:06.875695 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:47:06.875695 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:47:06.875695 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:47:06.875695 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:47:06.875695 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:47:06.875695 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:47:06.875695 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 10 11:47:06.875695 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 10 11:47:06.875695 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 10 11:47:06.875695 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 10 11:47:06.878158 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:47:06.878158 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:47:06.878158 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 10 11:47:06.878158 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 10 11:47:06.878158 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 10 11:47:06.878158 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 10 11:47:06.878158 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:47:06.878158 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:47:06.878158 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:47:06.878158 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:47:06.878158 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:47:06.878158 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application with Lock( Oct 10 11:47:06.878158 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:47:06.878158 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:47:06.878158 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:47:06.878158 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:47:06.878158 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:47:06.878158 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:47:06.879131 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:47:06.879131 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:47:06.879131 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 10 11:47:06.879131 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 10 11:47:06.879131 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 10 11:47:06.879131 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 10 11:47:06.879131 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 10 11:47:06.879131 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 10 11:47:06.879131 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Oct 10 11:47:06.879131 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Oct 10 11:47:06.879131 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 345/659] 10.222.0.46 () {56 vars in 1099 bytes} [Mon Oct 10 11:47:06 2022] GET /identity/v3/users/glance => generated 84 bytes in 307 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:47:06.880144 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901ce212-a098-4b6e-9f44-9b8ca3e0979f None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:06.915402 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b22e19-e30d-4c1e-9bec-cff80227f177 None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:06.958153 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b832cd-31ab-482e-8072-a4e6c30996ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:06.959226 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b832cd-31ab-482e-8072-a4e6c30996ab None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:06.959787 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b832cd-31ab-482e-8072-a4e6c30996ab None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:06.960270 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b832cd-31ab-482e-8072-a4e6c30996ab None admin] PATH_INFO: `/v3/users` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:06.961223 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b832cd-31ab-482e-8072-a4e6c30996ab None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:47:06.961755 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b832cd-31ab-482e-8072-a4e6c30996ab None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:06.963614 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b832cd-31ab-482e-8072-a4e6c30996ab None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:06.984947 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 315/660] 10.222.0.46 () {56 vars in 1133 bytes} [Mon Oct 10 11:47:06 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:47:06.986170 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b832cd-31ab-482e-8072-a4e6c30996ab None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:07.053527 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901ce212-a098-4b6e-9f44-9b8ca3e0979f None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:07.091134 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafee401-9a53-4cb8-8286-2040e4417d3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:07.092358 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafee401-9a53-4cb8-8286-2040e4417d3d None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:07.092639 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafee401-9a53-4cb8-8286-2040e4417d3d None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:07.092911 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafee401-9a53-4cb8-8286-2040e4417d3d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:07.101114 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dafee401-9a53-4cb8-8286-2040e4417d3d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:07.102638 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dafee401-9a53-4cb8-8286-2040e4417d3d None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:07.112672 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b832cd-31ab-482e-8072-a4e6c30996ab None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:07.115324 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application [None req-dafee401-9a53-4cb8-8286-2040e4417d3d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 10 11:47:07.115324 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:47:07.115324 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:47:07.115324 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:47:07.115324 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:47:07.115324 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:47:07.115324 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:47:07.115324 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:47:07.115324 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:47:07.115324 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:47:07.115324 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:47:07.115324 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:47:07.115324 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 10 11:47:07.115324 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 10 11:47:07.115324 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 10 11:47:07.116201 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 10 11:47:07.116201 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:47:07.116201 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:47:07.116201 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:47:07.116201 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:47:07.116201 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:47:07.116201 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:47:07.116201 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:47:07.116201 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application with Lock( Oct 10 11:47:07.116201 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:47:07.116201 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:47:07.116201 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:47:07.116201 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:47:07.116201 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:47:07.116201 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:47:07.116201 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:47:07.116201 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:47:07.117167 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 10 11:47:07.117167 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 10 11:47:07.117167 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 10 11:47:07.117167 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 10 11:47:07.117167 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 10 11:47:07.117167 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 10 11:47:07.117167 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 10 11:47:07.117167 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Oct 10 11:47:07.117167 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 346/661] 10.222.0.46 () {56 vars in 1111 bytes} [Mon Oct 10 11:47:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:47:07.118110 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafee401-9a53-4cb8-8286-2040e4417d3d None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:07.142242 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:07.159683 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d362e533-ddb5-4ecb-b22e-18f7128df421 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:07.160707 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d362e533-ddb5-4ecb-b22e-18f7128df421 None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:07.161224 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d362e533-ddb5-4ecb-b22e-18f7128df421 None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:07.161705 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d362e533-ddb5-4ecb-b22e-18f7128df421 None admin] PATH_INFO: `/v3/users` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:07.162706 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d362e533-ddb5-4ecb-b22e-18f7128df421 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:47:07.163360 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d362e533-ddb5-4ecb-b22e-18f7128df421 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:07.165064 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d362e533-ddb5-4ecb-b22e-18f7128df421 None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:07.170353 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:07.188263 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 316/662] 10.222.0.46 () {56 vars in 1137 bytes} [Mon Oct 10 11:47:06 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:47:07.189517 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d362e533-ddb5-4ecb-b22e-18f7128df421 None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:07.232926 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafee401-9a53-4cb8-8286-2040e4417d3d None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:07.270782 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517fe57e-cc76-4161-b44a-0539cd2d06ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:07.271495 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517fe57e-cc76-4161-b44a-0539cd2d06ef None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:07.271790 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517fe57e-cc76-4161-b44a-0539cd2d06ef None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:07.272029 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517fe57e-cc76-4161-b44a-0539cd2d06ef None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:07.310967 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d362e533-ddb5-4ecb-b22e-18f7128df421 None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:07.350902 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517fe57e-cc76-4161-b44a-0539cd2d06ef None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:07.351802 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710bc01a-b746-453e-b241-85caab5b9e04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:07.352565 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517fe57e-cc76-4161-b44a-0539cd2d06ef None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:07.353263 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710bc01a-b746-453e-b241-85caab5b9e04 None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:07.353758 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710bc01a-b746-453e-b241-85caab5b9e04 None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:07.354236 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710bc01a-b746-453e-b241-85caab5b9e04 None admin] PATH_INFO: `/v3/projects` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:07.355301 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710bc01a-b746-453e-b241-85caab5b9e04 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:47:07.355886 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710bc01a-b746-453e-b241-85caab5b9e04 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:07.357657 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710bc01a-b746-453e-b241-85caab5b9e04 None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:07.371764 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 317/663] 10.222.0.46 () {56 vars in 1148 bytes} [Mon Oct 10 11:47:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:47:07.373249 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710bc01a-b746-453e-b241-85caab5b9e04 None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:07.430525 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:07.444211 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:07.532795 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710bc01a-b746-453e-b241-85caab5b9e04 None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:07.567086 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 347/664] 10.222.0.46 () {58 vars in 1282 bytes} [Mon Oct 10 11:47:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 449 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 10 11:47:07.569643 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730bde28-9dfc-49a8-af2a-e4234b889b88 None None] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:07.569995 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730bde28-9dfc-49a8-af2a-e4234b889b88 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:07.570258 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730bde28-9dfc-49a8-af2a-e4234b889b88 None None] PATH_INFO: `/` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:07.571873 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 348/665] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 10 11:47:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:47:07.572381 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730bde28-9dfc-49a8-af2a-e4234b889b88 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:07.597065 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bff7bc-6dd7-4ea5-8f3e-23f4e2a404d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:07.598126 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bff7bc-6dd7-4ea5-8f3e-23f4e2a404d8 None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:07.598626 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bff7bc-6dd7-4ea5-8f3e-23f4e2a404d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:07.599114 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bff7bc-6dd7-4ea5-8f3e-23f4e2a404d8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:07.705613 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54bff7bc-6dd7-4ea5-8f3e-23f4e2a404d8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:07.707130 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54bff7bc-6dd7-4ea5-8f3e-23f4e2a404d8 None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:07.726175 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730bde28-9dfc-49a8-af2a-e4234b889b88 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:07.757960 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:07.795751 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3465ab6-fa6f-4706-a1f4-b87dfac916b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:07.796822 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3465ab6-fa6f-4706-a1f4-b87dfac916b9 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:07.797288 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3465ab6-fa6f-4706-a1f4-b87dfac916b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:07.797876 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3465ab6-fa6f-4706-a1f4-b87dfac916b9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:07.799191 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3465ab6-fa6f-4706-a1f4-b87dfac916b9 None admin] RBAC: Adding query filter params (role.id=3c35be38787740f4834c728a2ce73b6b, scope.project.id=4ad3772f7b6a47c28b7ba37a9316a3c1, user.id=58b1c7c85302421cba823ace4201861f) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:47:07.799747 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3465ab6-fa6f-4706-a1f4-b87dfac916b9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:07.802504 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3465ab6-fa6f-4706-a1f4-b87dfac916b9 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:07.814338 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 349/666] 10.222.0.46 () {56 vars in 1374 bytes} [Mon Oct 10 11:47:07 2022] GET /identity/v3/role_assignments?user.id=58b1c7c85302421cba823ace4201861f&scope.project.id=4ad3772f7b6a47c28b7ba37a9316a3c1&role.id=3c35be38787740f4834c728a2ce73b6b => generated 608 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:47:07.815954 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3465ab6-fa6f-4706-a1f4-b87dfac916b9 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:07.840841 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:07.870734 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 318/667] 10.222.0.46 () {58 vars in 1282 bytes} [Mon Oct 10 11:47:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 498 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 10 11:47:07.873603 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c71089c-087c-452a-92f6-8dd53279a7ff None None] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:07.874136 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c71089c-087c-452a-92f6-8dd53279a7ff None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:07.874628 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c71089c-087c-452a-92f6-8dd53279a7ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:07.907373 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.auth.core [None req-8c71089c-087c-452a-92f6-8dd53279a7ff None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:47:07.909470 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-8c71089c-087c-452a-92f6-8dd53279a7ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:47:08.043635 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3465ab6-fa6f-4706-a1f4-b87dfac916b9 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:08.063182 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 319/668] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 10 11:47:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 10 11:47:08.066240 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c71089c-087c-452a-92f6-8dd53279a7ff None None] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:08.116610 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a80f08-cfcd-4588-8218-3bad94821c6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:08.117322 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a80f08-cfcd-4588-8218-3bad94821c6e None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:08.117630 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a80f08-cfcd-4588-8218-3bad94821c6e None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:08.117963 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a80f08-cfcd-4588-8218-3bad94821c6e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:08.142526 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a80f08-cfcd-4588-8218-3bad94821c6e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:08.143779 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a80f08-cfcd-4588-8218-3bad94821c6e None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:08.175596 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application [None req-62a80f08-cfcd-4588-8218-3bad94821c6e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:47:08.175596 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:47:08.175596 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:47:08.175596 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:47:08.175596 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:47:08.175596 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:47:08.175596 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:47:08.175596 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:47:08.175596 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:47:08.175596 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:47:08.175596 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:47:08.175596 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:47:08.175596 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 10 11:47:08.175596 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 10 11:47:08.175596 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 10 11:47:08.175596 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 10 11:47:08.177418 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:47:08.177418 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:47:08.177418 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:47:08.177418 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:47:08.177418 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:47:08.177418 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:47:08.177418 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:47:08.177418 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application with Lock( Oct 10 11:47:08.177418 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:47:08.177418 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:47:08.177418 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:47:08.177418 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:47:08.177418 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:47:08.177418 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:47:08.177418 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:47:08.177418 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:47:08.177418 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 10 11:47:08.179203 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 10 11:47:08.179203 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:47:08.179203 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Oct 10 11:47:08.179203 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 350/669] 10.222.0.46 () {56 vars in 1108 bytes} [Mon Oct 10 11:47:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 364 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:47:08.189761 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a80f08-cfcd-4588-8218-3bad94821c6e None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:08.225320 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:08.286466 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c71089c-087c-452a-92f6-8dd53279a7ff None None] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:08.332292 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:08.349650 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e41e22b-5539-4df3-b2bc-e77bee28fdcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:08.350305 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e41e22b-5539-4df3-b2bc-e77bee28fdcf None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:08.352787 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e41e22b-5539-4df3-b2bc-e77bee28fdcf None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:08.352787 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e41e22b-5539-4df3-b2bc-e77bee28fdcf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:08.392619 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e41e22b-5539-4df3-b2bc-e77bee28fdcf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:08.395394 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e41e22b-5539-4df3-b2bc-e77bee28fdcf None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:08.430109 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application [None req-8e41e22b-5539-4df3-b2bc-e77bee28fdcf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:47:08.430109 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:47:08.430109 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:47:08.430109 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:47:08.430109 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:47:08.430109 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:47:08.430109 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:47:08.430109 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:47:08.430109 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:47:08.430109 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:47:08.430109 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:47:08.430109 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:47:08.430109 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 10 11:47:08.430109 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 10 11:47:08.430109 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 10 11:47:08.430109 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 10 11:47:08.432068 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:47:08.432068 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:47:08.432068 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:47:08.432068 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:47:08.432068 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:47:08.432068 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:47:08.432068 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:47:08.432068 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application with Lock( Oct 10 11:47:08.432068 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:47:08.432068 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:47:08.432068 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:47:08.432068 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:47:08.432068 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:47:08.432068 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:47:08.432068 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:47:08.432068 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:47:08.432068 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 10 11:47:08.433890 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 10 11:47:08.433890 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:47:08.433890 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Oct 10 11:47:08.433890 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 320/670] 10.222.0.46 () {56 vars in 1108 bytes} [Mon Oct 10 11:47:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 369 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:47:08.442822 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e41e22b-5539-4df3-b2bc-e77bee28fdcf None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:08.477879 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a80f08-cfcd-4588-8218-3bad94821c6e None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:08.580599 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55f6280-5e19-407e-9e2b-c284a3eb4b8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:08.581620 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55f6280-5e19-407e-9e2b-c284a3eb4b8a None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:08.581620 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55f6280-5e19-407e-9e2b-c284a3eb4b8a None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:08.581869 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55f6280-5e19-407e-9e2b-c284a3eb4b8a None admin] PATH_INFO: `/v3/domains` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:08.582492 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55f6280-5e19-407e-9e2b-c284a3eb4b8a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:47:08.582795 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55f6280-5e19-407e-9e2b-c284a3eb4b8a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:08.584136 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55f6280-5e19-407e-9e2b-c284a3eb4b8a None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:08.610176 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 351/671] 10.222.0.46 () {56 vars in 1109 bytes} [Mon Oct 10 11:47:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 421 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 11:47:08.611426 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55f6280-5e19-407e-9e2b-c284a3eb4b8a None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:08.645138 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e41e22b-5539-4df3-b2bc-e77bee28fdcf None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:08.669775 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:08.688181 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9482f03-6327-4200-9233-99d34d6e2b67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:08.689477 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9482f03-6327-4200-9233-99d34d6e2b67 None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:08.689948 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9482f03-6327-4200-9233-99d34d6e2b67 None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:08.690478 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9482f03-6327-4200-9233-99d34d6e2b67 None admin] PATH_INFO: `/v3/domains` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:08.691350 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9482f03-6327-4200-9233-99d34d6e2b67 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:47:08.691855 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9482f03-6327-4200-9233-99d34d6e2b67 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:08.694005 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9482f03-6327-4200-9233-99d34d6e2b67 None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:08.711317 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 321/672] 10.222.0.46 () {56 vars in 1109 bytes} [Mon Oct 10 11:47:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:47:08.712862 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9482f03-6327-4200-9233-99d34d6e2b67 None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:08.758168 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55f6280-5e19-407e-9e2b-c284a3eb4b8a None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:08.789442 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:08.813220 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e1bb84-205c-4723-8aea-0d4b87baca3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:08.814609 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e1bb84-205c-4723-8aea-0d4b87baca3c None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:08.815106 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e1bb84-205c-4723-8aea-0d4b87baca3c None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:08.815596 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e1bb84-205c-4723-8aea-0d4b87baca3c None admin] PATH_INFO: `/v3/roles/service` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:08.837814 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e1bb84-205c-4723-8aea-0d4b87baca3c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 10 11:47:08.847682 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application [None req-13e1bb84-205c-4723-8aea-0d4b87baca3c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 10 11:47:08.847682 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:47:08.847682 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:47:08.847682 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:47:08.847682 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:47:08.847682 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:47:08.847682 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:47:08.847682 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:47:08.847682 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:47:08.847682 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:47:08.847682 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:47:08.847682 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:47:08.847682 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 10 11:47:08.847682 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 10 11:47:08.847682 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 10 11:47:08.847682 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 10 11:47:08.848817 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 10 11:47:08.848817 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 10 11:47:08.848817 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 10 11:47:08.848817 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 10 11:47:08.848817 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:47:08.848817 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:47:08.848817 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:47:08.848817 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:47:08.848817 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:47:08.848817 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:47:08.848817 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:47:08.848817 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application with Lock( Oct 10 11:47:08.848817 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:47:08.848817 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:47:08.848817 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:47:08.848817 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:47:08.849640 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:47:08.849640 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:47:08.849640 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:47:08.849640 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:47:08.849640 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 10 11:47:08.849640 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 10 11:47:08.849640 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 10 11:47:08.849640 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 10 11:47:08.849640 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 10 11:47:08.849640 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 10 11:47:08.849640 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 10 11:47:08.849640 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Oct 10 11:47:08.850269 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 352/673] 10.222.0.46 () {56 vars in 1102 bytes} [Mon Oct 10 11:47:08 2022] GET /identity/v3/roles/service => generated 85 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:47:08.851612 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e1bb84-205c-4723-8aea-0d4b87baca3c None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:08.889184 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9482f03-6327-4200-9233-99d34d6e2b67 None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:08.941571 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccd01d9-9d72-461a-8028-ee0a8827ce64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:08.942336 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccd01d9-9d72-461a-8028-ee0a8827ce64 None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:08.942619 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccd01d9-9d72-461a-8028-ee0a8827ce64 None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:08.942959 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccd01d9-9d72-461a-8028-ee0a8827ce64 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:08.953127 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ccd01d9-9d72-461a-8028-ee0a8827ce64 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:08.954701 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ccd01d9-9d72-461a-8028-ee0a8827ce64 None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:08.967424 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application [None req-2ccd01d9-9d72-461a-8028-ee0a8827ce64 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 10 11:47:08.967424 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:47:08.967424 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:47:08.967424 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:47:08.967424 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:47:08.967424 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:47:08.967424 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:47:08.967424 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:47:08.967424 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:47:08.967424 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:47:08.967424 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:47:08.967424 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:47:08.967424 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 10 11:47:08.967424 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 10 11:47:08.967424 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 10 11:47:08.968851 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 10 11:47:08.968851 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:47:08.968851 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:47:08.968851 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:47:08.968851 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:47:08.968851 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:47:08.968851 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:47:08.968851 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:47:08.968851 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application with Lock( Oct 10 11:47:08.968851 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:47:08.968851 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:47:08.968851 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:47:08.968851 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:47:08.968851 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:47:08.968851 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:47:08.968851 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:47:08.968851 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:47:08.970564 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 10 11:47:08.970564 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 10 11:47:08.970564 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 10 11:47:08.970564 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 10 11:47:08.970564 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 10 11:47:08.970564 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 10 11:47:08.970564 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 10 11:47:08.970564 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Oct 10 11:47:08.970564 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 322/674] 10.222.0.46 () {56 vars in 1111 bytes} [Mon Oct 10 11:47:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:47:08.970564 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed246dc-ba48-46d3-a927-366648ed1141 None None] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:08.970564 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed246dc-ba48-46d3-a927-366648ed1141 None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:08.972514 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed246dc-ba48-46d3-a927-366648ed1141 None None] PATH_INFO: `/` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:08.976205 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 323/675] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 10 11:47:08 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:47:08.977134 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed246dc-ba48-46d3-a927-366648ed1141 None None] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:09.012075 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e1bb84-205c-4723-8aea-0d4b87baca3c None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:09.032853 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:09.048969 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d079866-2e50-49ec-87eb-cbe0da577cba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:09.049617 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d079866-2e50-49ec-87eb-cbe0da577cba None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:09.049896 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d079866-2e50-49ec-87eb-cbe0da577cba None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:09.050166 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d079866-2e50-49ec-87eb-cbe0da577cba None admin] PATH_INFO: `/v3/projects/service` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:09.057666 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d079866-2e50-49ec-87eb-cbe0da577cba None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:09.059169 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d079866-2e50-49ec-87eb-cbe0da577cba None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:09.071853 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application [None req-0d079866-2e50-49ec-87eb-cbe0da577cba None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 10 11:47:09.071853 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:47:09.071853 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:47:09.071853 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:47:09.071853 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:47:09.071853 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:47:09.071853 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:47:09.071853 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:47:09.071853 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:47:09.071853 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:47:09.071853 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:47:09.071853 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:47:09.071853 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 10 11:47:09.071853 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 10 11:47:09.071853 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 10 11:47:09.072818 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 10 11:47:09.072818 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:47:09.072818 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:47:09.072818 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:47:09.072818 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:47:09.072818 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:47:09.072818 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:47:09.072818 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:47:09.072818 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application with Lock( Oct 10 11:47:09.072818 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:47:09.072818 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:47:09.072818 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:47:09.072818 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:47:09.072818 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:47:09.072818 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:47:09.072818 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:47:09.072818 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:47:09.073679 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 10 11:47:09.073679 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 10 11:47:09.073679 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 10 11:47:09.073679 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 10 11:47:09.073679 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 10 11:47:09.073679 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 10 11:47:09.073679 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 10 11:47:09.073679 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Oct 10 11:47:09.077898 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 353/676] 10.222.0.46 () {56 vars in 1111 bytes} [Mon Oct 10 11:47:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:47:09.079321 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b96f6a-9030-4463-ad60-787dc21236e9 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:09.079623 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b96f6a-9030-4463-ad60-787dc21236e9 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:09.079898 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b96f6a-9030-4463-ad60-787dc21236e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:09.098948 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-16b96f6a-9030-4463-ad60-787dc21236e9 None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:47:09.100074 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-16b96f6a-9030-4463-ad60-787dc21236e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:47:09.138106 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:09.149598 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed246dc-ba48-46d3-a927-366648ed1141 None None] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:09.186339 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d7949b-57dc-4284-bc11-71e1d33cafb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:09.187054 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d7949b-57dc-4284-bc11-71e1d33cafb2 None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:09.187432 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d7949b-57dc-4284-bc11-71e1d33cafb2 None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:09.187622 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d7949b-57dc-4284-bc11-71e1d33cafb2 None admin] PATH_INFO: `/v3/roles` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:09.188321 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d7949b-57dc-4284-bc11-71e1d33cafb2 None admin] RBAC: Adding query filter params (name=service) {{(pid=81676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:47:09.188626 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d7949b-57dc-4284-bc11-71e1d33cafb2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:09.194705 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 354/677] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 10 11:47:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 10 11:47:09.194888 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d7949b-57dc-4284-bc11-71e1d33cafb2 None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:09.195005 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b96f6a-9030-4463-ad60-787dc21236e9 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:09.204623 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 324/678] 10.222.0.46 () {56 vars in 1103 bytes} [Mon Oct 10 11:47:08 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:47:09.205598 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d7949b-57dc-4284-bc11-71e1d33cafb2 None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:09.319962 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b96f6a-9030-4463-ad60-787dc21236e9 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:09.350470 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d7949b-57dc-4284-bc11-71e1d33cafb2 None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:09.353908 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:09.361805 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71773d1-480d-4050-932b-a33125b5dc66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:09.363025 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71773d1-480d-4050-932b-a33125b5dc66 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:09.363312 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71773d1-480d-4050-932b-a33125b5dc66 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:09.363594 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71773d1-480d-4050-932b-a33125b5dc66 None admin] PATH_INFO: `/v3/projects` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:09.370829 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71773d1-480d-4050-932b-a33125b5dc66 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:47:09.370829 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71773d1-480d-4050-932b-a33125b5dc66 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:09.370829 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71773d1-480d-4050-932b-a33125b5dc66 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:09.386835 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 355/679] 10.222.0.46 () {56 vars in 1148 bytes} [Mon Oct 10 11:47:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:47:09.388107 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71773d1-480d-4050-932b-a33125b5dc66 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:09.402190 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be39687-8768-4251-a23c-154051bc345a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:09.402880 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be39687-8768-4251-a23c-154051bc345a None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:09.403148 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be39687-8768-4251-a23c-154051bc345a None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:09.403424 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be39687-8768-4251-a23c-154051bc345a None admin] PATH_INFO: `/v3/projects` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:09.404176 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be39687-8768-4251-a23c-154051bc345a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:47:09.404486 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be39687-8768-4251-a23c-154051bc345a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:09.405932 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be39687-8768-4251-a23c-154051bc345a None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:09.430201 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 325/680] 10.222.0.46 () {56 vars in 1148 bytes} [Mon Oct 10 11:47:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:47:09.430201 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be39687-8768-4251-a23c-154051bc345a None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:09.446997 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:09.567644 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71773d1-480d-4050-932b-a33125b5dc66 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:09.595627 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be39687-8768-4251-a23c-154051bc345a None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:09.634059 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d1a8ef-4db8-49c2-a945-61f5ecd60467 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:09.635198 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d1a8ef-4db8-49c2-a945-61f5ecd60467 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:09.635691 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d1a8ef-4db8-49c2-a945-61f5ecd60467 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:09.636131 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d1a8ef-4db8-49c2-a945-61f5ecd60467 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:09.637734 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98a8289-0ff9-4609-a413-4a17080ad295 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:09.638489 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98a8289-0ff9-4609-a413-4a17080ad295 None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:09.638892 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98a8289-0ff9-4609-a413-4a17080ad295 None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:09.639450 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98a8289-0ff9-4609-a413-4a17080ad295 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:09.640130 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98a8289-0ff9-4609-a413-4a17080ad295 None admin] RBAC: Adding query filter params (role.id=3c35be38787740f4834c728a2ce73b6b, scope.project.id=4ad3772f7b6a47c28b7ba37a9316a3c1, user.id=5f130bbb69e546ce99b4e14de81d7e54) {{(pid=81676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:47:09.640546 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98a8289-0ff9-4609-a413-4a17080ad295 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:09.642106 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98a8289-0ff9-4609-a413-4a17080ad295 None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:09.648972 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 326/681] 10.222.0.46 () {56 vars in 1374 bytes} [Mon Oct 10 11:47:09 2022] GET /identity/v3/role_assignments?user.id=5f130bbb69e546ce99b4e14de81d7e54&scope.project.id=4ad3772f7b6a47c28b7ba37a9316a3c1&role.id=3c35be38787740f4834c728a2ce73b6b => generated 608 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:47:09.649932 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98a8289-0ff9-4609-a413-4a17080ad295 None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:09.672407 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d1a8ef-4db8-49c2-a945-61f5ecd60467 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:09.675334 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d1a8ef-4db8-49c2-a945-61f5ecd60467 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:09.704328 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application [None req-94d1a8ef-4db8-49c2-a945-61f5ecd60467 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:47:09.704328 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:47:09.704328 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:47:09.704328 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:47:09.704328 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:47:09.704328 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:47:09.704328 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:47:09.704328 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:47:09.704328 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:47:09.704328 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:47:09.704328 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:47:09.704328 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:47:09.704328 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 10 11:47:09.704328 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 10 11:47:09.704328 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 10 11:47:09.704328 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 10 11:47:09.709329 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:47:09.709329 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:47:09.709329 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:47:09.709329 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:47:09.709329 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:47:09.709329 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:47:09.709329 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:47:09.709329 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application with Lock( Oct 10 11:47:09.709329 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:47:09.709329 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:47:09.709329 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:47:09.709329 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:47:09.709329 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:47:09.709329 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:47:09.709329 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:47:09.709329 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:47:09.709329 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 10 11:47:09.711263 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 10 11:47:09.711263 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:47:09.711263 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Oct 10 11:47:09.714368 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:09.714582 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 356/682] 10.222.0.46 () {56 vars in 1108 bytes} [Mon Oct 10 11:47:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 326 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:47:09.716931 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d1a8ef-4db8-49c2-a945-61f5ecd60467 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:09.868355 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98a8289-0ff9-4609-a413-4a17080ad295 None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:09.893839 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:09.934759 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d1a8ef-4db8-49c2-a945-61f5ecd60467 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:09.946558 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf64c8bb-cdbd-4e39-a26d-6eb27142cddd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:09.947711 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf64c8bb-cdbd-4e39-a26d-6eb27142cddd None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:09.948171 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf64c8bb-cdbd-4e39-a26d-6eb27142cddd None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:09.948656 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf64c8bb-cdbd-4e39-a26d-6eb27142cddd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:09.950080 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf64c8bb-cdbd-4e39-a26d-6eb27142cddd None admin] RBAC: Adding query filter params (role.id=3c35be38787740f4834c728a2ce73b6b, scope.project.id=4ad3772f7b6a47c28b7ba37a9316a3c1, user.id=124f4aa8ab244ca2aadf4f2614df51e7) {{(pid=81676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:47:09.950657 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf64c8bb-cdbd-4e39-a26d-6eb27142cddd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:09.953794 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf64c8bb-cdbd-4e39-a26d-6eb27142cddd None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:09.971258 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 327/683] 10.222.0.46 () {56 vars in 1374 bytes} [Mon Oct 10 11:47:09 2022] GET /identity/v3/role_assignments?user.id=124f4aa8ab244ca2aadf4f2614df51e7&scope.project.id=4ad3772f7b6a47c28b7ba37a9316a3c1&role.id=3c35be38787740f4834c728a2ce73b6b => generated 608 bytes in 321 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:47:09.973076 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf64c8bb-cdbd-4e39-a26d-6eb27142cddd None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:10.001266 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d92e7e1-47b9-426b-8b34-b5fe0afc676a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:10.002053 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d92e7e1-47b9-426b-8b34-b5fe0afc676a None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:10.002363 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d92e7e1-47b9-426b-8b34-b5fe0afc676a None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:10.002612 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d92e7e1-47b9-426b-8b34-b5fe0afc676a None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:10.003246 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d92e7e1-47b9-426b-8b34-b5fe0afc676a None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:10.004686 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d92e7e1-47b9-426b-8b34-b5fe0afc676a None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:10.019484 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application [None req-1d92e7e1-47b9-426b-8b34-b5fe0afc676a None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 10 11:47:10.019484 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:47:10.019484 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 10 11:47:10.019484 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 10 11:47:10.019484 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 10 11:47:10.019484 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 10 11:47:10.019484 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 10 11:47:10.019484 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 10 11:47:10.019484 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 10 11:47:10.019484 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Oct 10 11:47:10.019484 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 10 11:47:10.019484 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Oct 10 11:47:10.019484 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:47:10.019484 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:47:10.019484 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:47:10.019484 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:47:10.019484 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:47:10.021379 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:47:10.021379 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:47:10.021379 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:47:10.021379 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:47:10.021379 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:47:10.021379 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:47:10.021379 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 10 11:47:10.021379 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 10 11:47:10.021379 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 10 11:47:10.021379 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 10 11:47:10.021379 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:47:10.021379 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:47:10.021379 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:47:10.021379 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:47:10.021379 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:47:10.021379 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:47:10.023009 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:47:10.023009 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application with Lock( Oct 10 11:47:10.023009 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:47:10.023009 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:47:10.023009 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:47:10.023009 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:47:10.023009 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:47:10.023009 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:47:10.023009 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:47:10.023009 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:47:10.023009 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 10 11:47:10.023009 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 10 11:47:10.023009 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 10 11:47:10.023009 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Oct 10 11:47:10.023009 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 357/684] 10.222.0.46 () {56 vars in 1129 bytes} [Mon Oct 10 11:47:09 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 306 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:47:10.036711 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d92e7e1-47b9-426b-8b34-b5fe0afc676a None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:10.197901 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:10.222574 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf64c8bb-cdbd-4e39-a26d-6eb27142cddd None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:10.287869 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85823dc2-1fe4-4bda-8cea-71fbbbc3ae10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:10.288503 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85823dc2-1fe4-4bda-8cea-71fbbbc3ae10 None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:10.288781 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85823dc2-1fe4-4bda-8cea-71fbbbc3ae10 None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:10.289053 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85823dc2-1fe4-4bda-8cea-71fbbbc3ae10 None admin] PATH_INFO: `/v3/domains` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:10.289642 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d92e7e1-47b9-426b-8b34-b5fe0afc676a None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:10.289805 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85823dc2-1fe4-4bda-8cea-71fbbbc3ae10 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:47:10.290055 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85823dc2-1fe4-4bda-8cea-71fbbbc3ae10 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:10.291478 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85823dc2-1fe4-4bda-8cea-71fbbbc3ae10 None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:10.309460 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 328/685] 10.222.0.46 () {56 vars in 1109 bytes} [Mon Oct 10 11:47:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 333 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:47:10.311476 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85823dc2-1fe4-4bda-8cea-71fbbbc3ae10 None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:10.359417 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d165ed72-af73-4109-8906-f29f71964a84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:10.360976 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d165ed72-af73-4109-8906-f29f71964a84 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:10.361884 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d165ed72-af73-4109-8906-f29f71964a84 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:10.362643 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d165ed72-af73-4109-8906-f29f71964a84 None admin] PATH_INFO: `/v3/services` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:10.364185 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d165ed72-af73-4109-8906-f29f71964a84 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:47:10.365055 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d165ed72-af73-4109-8906-f29f71964a84 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:10.367944 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d165ed72-af73-4109-8906-f29f71964a84 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:10.379165 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 358/686] 10.222.0.46 () {56 vars in 1124 bytes} [Mon Oct 10 11:47:10 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 343 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 11:47:10.386504 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d165ed72-af73-4109-8906-f29f71964a84 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:10.402267 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:10.459928 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85823dc2-1fe4-4bda-8cea-71fbbbc3ae10 None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:10.524102 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e290497a-d459-457e-8597-5347f1a3042a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:10.526033 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e290497a-d459-457e-8597-5347f1a3042a None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:10.527263 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e290497a-d459-457e-8597-5347f1a3042a None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:10.528023 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e290497a-d459-457e-8597-5347f1a3042a None admin] PATH_INFO: `/v3/users/swift` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:10.537028 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d165ed72-af73-4109-8906-f29f71964a84 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:10.543792 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e290497a-d459-457e-8597-5347f1a3042a None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:10.547709 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e290497a-d459-457e-8597-5347f1a3042a None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:10.572601 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application [None req-e290497a-d459-457e-8597-5347f1a3042a None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 10 11:47:10.572601 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:47:10.572601 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:47:10.572601 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:47:10.572601 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:47:10.572601 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:47:10.572601 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:47:10.572601 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:47:10.572601 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:47:10.572601 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:47:10.572601 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:47:10.572601 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:47:10.572601 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 10 11:47:10.572601 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 10 11:47:10.572601 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 10 11:47:10.572601 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 10 11:47:10.574401 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:47:10.574401 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:47:10.574401 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 10 11:47:10.574401 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 10 11:47:10.574401 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 10 11:47:10.574401 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 10 11:47:10.574401 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:47:10.574401 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:47:10.574401 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:47:10.574401 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:47:10.574401 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:47:10.574401 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application with Lock( Oct 10 11:47:10.574401 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:47:10.574401 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:47:10.574401 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:47:10.574401 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:47:10.574401 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:47:10.574401 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:47:10.579731 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:47:10.579731 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:47:10.579731 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 10 11:47:10.579731 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 10 11:47:10.579731 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 10 11:47:10.579731 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 10 11:47:10.579731 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 10 11:47:10.579731 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 10 11:47:10.579731 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 10 11:47:10.579731 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Oct 10 11:47:10.579731 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 329/687] 10.222.0.46 () {56 vars in 1096 bytes} [Mon Oct 10 11:47:10 2022] GET /identity/v3/users/swift => generated 83 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 10 11:47:10.582892 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e290497a-d459-457e-8597-5347f1a3042a None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:10.587176 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dccdd91-c19c-4fd1-bd39-0aa55c13d1a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:10.587810 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dccdd91-c19c-4fd1-bd39-0aa55c13d1a0 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:10.588095 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dccdd91-c19c-4fd1-bd39-0aa55c13d1a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:10.588371 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dccdd91-c19c-4fd1-bd39-0aa55c13d1a0 None admin] PATH_INFO: `/v3/services` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:10.589085 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dccdd91-c19c-4fd1-bd39-0aa55c13d1a0 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:47:10.589391 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dccdd91-c19c-4fd1-bd39-0aa55c13d1a0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:10.590958 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dccdd91-c19c-4fd1-bd39-0aa55c13d1a0 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:10.599147 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 359/688] 10.222.0.46 () {56 vars in 1124 bytes} [Mon Oct 10 11:47:10 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:47:10.600441 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40753fb0-95a7-41d7-a453-142cfe490f27 None None] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:10.600789 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40753fb0-95a7-41d7-a453-142cfe490f27 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:10.601082 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40753fb0-95a7-41d7-a453-142cfe490f27 None None] PATH_INFO: `/` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:10.604059 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 360/689] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 10 11:47:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:47:10.617920 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8b6866-5cb9-4e51-ba2c-11f0291e27c2 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:10.618328 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8b6866-5cb9-4e51-ba2c-11f0291e27c2 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:10.618479 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8b6866-5cb9-4e51-ba2c-11f0291e27c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:10.623327 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:10.663882 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-5a8b6866-5cb9-4e51-ba2c-11f0291e27c2 None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:47:10.665104 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-5a8b6866-5cb9-4e51-ba2c-11f0291e27c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:47:10.821747 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e290497a-d459-457e-8597-5347f1a3042a None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:10.837192 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 361/690] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 10 11:47:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 10 11:47:10.875025 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88df4398-7f95-4799-b6be-93d26401f32a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:10.875652 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88df4398-7f95-4799-b6be-93d26401f32a None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:10.875928 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88df4398-7f95-4799-b6be-93d26401f32a None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:10.876208 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88df4398-7f95-4799-b6be-93d26401f32a None admin] PATH_INFO: `/v3/users` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:10.876967 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88df4398-7f95-4799-b6be-93d26401f32a None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:47:10.877277 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88df4398-7f95-4799-b6be-93d26401f32a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:10.878928 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88df4398-7f95-4799-b6be-93d26401f32a None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:10.917147 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 330/691] 10.222.0.46 () {56 vars in 1135 bytes} [Mon Oct 10 11:47:10 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 335 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:47:10.927718 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8b6866-5cb9-4e51-ba2c-11f0291e27c2 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:10.935347 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:11.127318 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8b6866-5cb9-4e51-ba2c-11f0291e27c2 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:11.197008 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5d492f-4a7d-420a-b8e9-b3550502b224 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:11.198087 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5d492f-4a7d-420a-b8e9-b3550502b224 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:11.198700 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5d492f-4a7d-420a-b8e9-b3550502b224 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:11.199281 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5d492f-4a7d-420a-b8e9-b3550502b224 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:11.351994 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5d492f-4a7d-420a-b8e9-b3550502b224 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:11.354957 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5d492f-4a7d-420a-b8e9-b3550502b224 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:11.360170 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:11.381966 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88df4398-7f95-4799-b6be-93d26401f32a None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:11.598834 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 362/692] 10.222.0.46 () {58 vars in 1282 bytes} [Mon Oct 10 11:47:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 672 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 10 11:47:11.607737 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5d492f-4a7d-420a-b8e9-b3550502b224 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:11.611154 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:11.662329 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88df4398-7f95-4799-b6be-93d26401f32a None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:11.724852 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a224dc-b1a7-460a-bbe4-d4607001e089 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:11.725934 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a224dc-b1a7-460a-bbe4-d4607001e089 None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:11.726426 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a224dc-b1a7-460a-bbe4-d4607001e089 None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:11.726926 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a224dc-b1a7-460a-bbe4-d4607001e089 None admin] PATH_INFO: `/v3/services/network` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:11.727990 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a224dc-b1a7-460a-bbe4-d4607001e089 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:11.736279 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a224dc-b1a7-460a-bbe4-d4607001e089 None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:11.775583 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application [None req-01a224dc-b1a7-460a-bbe4-d4607001e089 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Oct 10 11:47:11.775583 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:47:11.775583 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 10 11:47:11.775583 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 10 11:47:11.775583 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 10 11:47:11.775583 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 10 11:47:11.775583 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 10 11:47:11.775583 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 10 11:47:11.775583 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 10 11:47:11.775583 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Oct 10 11:47:11.775583 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 10 11:47:11.775583 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Oct 10 11:47:11.775583 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:47:11.775583 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:47:11.775583 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:47:11.775583 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:47:11.775583 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:47:11.776649 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:47:11.776649 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:47:11.776649 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:47:11.776649 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:47:11.776649 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:47:11.776649 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:47:11.776649 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 10 11:47:11.776649 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 10 11:47:11.776649 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 10 11:47:11.776649 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 10 11:47:11.776649 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:47:11.776649 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:47:11.776649 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:47:11.776649 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:47:11.776649 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:47:11.776649 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:47:11.777554 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:47:11.777554 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application with Lock( Oct 10 11:47:11.777554 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:47:11.777554 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:47:11.777554 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:47:11.777554 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:47:11.777554 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:47:11.777554 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:47:11.777554 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:47:11.777554 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:47:11.777554 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 10 11:47:11.777554 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 10 11:47:11.777554 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 10 11:47:11.777554 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Oct 10 11:47:11.786691 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 331/693] 10.222.0.46 () {56 vars in 1111 bytes} [Mon Oct 10 11:47:11 2022] GET /identity/v3/services/network => generated 88 bytes in 396 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 10 11:47:11.788223 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a224dc-b1a7-460a-bbe4-d4607001e089 None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:11.912779 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5d492f-4a7d-420a-b8e9-b3550502b224 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:11.916846 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:12.016516 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a224dc-b1a7-460a-bbe4-d4607001e089 None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:12.047299 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f566b2-cdb9-4313-be14-45dc13984d7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:12.048456 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f566b2-cdb9-4313-be14-45dc13984d7a None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:12.048735 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f566b2-cdb9-4313-be14-45dc13984d7a None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:12.049003 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f566b2-cdb9-4313-be14-45dc13984d7a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:12.076413 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f566b2-cdb9-4313-be14-45dc13984d7a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:12.078815 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f566b2-cdb9-4313-be14-45dc13984d7a None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:12.099313 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:12.119611 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application [None req-e5f566b2-cdb9-4313-be14-45dc13984d7a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:47:12.119611 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:47:12.119611 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:47:12.119611 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:47:12.119611 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:47:12.119611 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:47:12.119611 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:47:12.119611 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:47:12.119611 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:47:12.119611 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:47:12.119611 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:47:12.119611 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:47:12.119611 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 10 11:47:12.119611 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 10 11:47:12.119611 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 10 11:47:12.119611 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 10 11:47:12.121738 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:47:12.121738 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:47:12.121738 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:47:12.121738 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:47:12.121738 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:47:12.121738 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:47:12.121738 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:47:12.121738 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application with Lock( Oct 10 11:47:12.121738 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:47:12.121738 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:47:12.121738 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:47:12.121738 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:47:12.121738 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:47:12.121738 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:47:12.121738 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:47:12.121738 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:47:12.121738 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 10 11:47:12.124450 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 10 11:47:12.124450 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:47:12.124450 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Oct 10 11:47:12.128804 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814f87f9-5cc4-4cd9-ab4d-0cb79043f836 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:12.129806 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814f87f9-5cc4-4cd9-ab4d-0cb79043f836 None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:12.130313 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814f87f9-5cc4-4cd9-ab4d-0cb79043f836 None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:12.130834 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814f87f9-5cc4-4cd9-ab4d-0cb79043f836 None admin] PATH_INFO: `/v3/services` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:12.131967 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814f87f9-5cc4-4cd9-ab4d-0cb79043f836 None admin] RBAC: Adding query filter params (name=network) {{(pid=81676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:47:12.132516 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814f87f9-5cc4-4cd9-ab4d-0cb79043f836 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:12.139141 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814f87f9-5cc4-4cd9-ab4d-0cb79043f836 None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:12.141279 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 363/694] 10.222.0.46 () {56 vars in 1108 bytes} [Mon Oct 10 11:47:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 519 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 10 11:47:12.142649 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f566b2-cdb9-4313-be14-45dc13984d7a None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:12.152772 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 332/695] 10.222.0.46 () {56 vars in 1112 bytes} [Mon Oct 10 11:47:11 2022] GET /identity/v3/services?name=network => generated 124 bytes in 365 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:47:12.166626 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814f87f9-5cc4-4cd9-ab4d-0cb79043f836 None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:12.337353 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814f87f9-5cc4-4cd9-ab4d-0cb79043f836 None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:12.398809 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f566b2-cdb9-4313-be14-45dc13984d7a None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:12.403140 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcee567b-179e-41fb-9a83-79f61892d226 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:12.404069 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcee567b-179e-41fb-9a83-79f61892d226 None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:12.405525 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcee567b-179e-41fb-9a83-79f61892d226 None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:12.406001 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcee567b-179e-41fb-9a83-79f61892d226 None admin] PATH_INFO: `/v3/services` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:12.408917 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcee567b-179e-41fb-9a83-79f61892d226 None admin] RBAC: Adding query filter params (type=network) {{(pid=81676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:47:12.409470 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcee567b-179e-41fb-9a83-79f61892d226 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:12.412016 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcee567b-179e-41fb-9a83-79f61892d226 None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:12.441886 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 333/696] 10.222.0.46 () {56 vars in 1112 bytes} [Mon Oct 10 11:47:12 2022] GET /identity/v3/services?type=network => generated 124 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:47:12.444676 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9c311d-484b-43dd-95da-537edf975138 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:12.445770 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9c311d-484b-43dd-95da-537edf975138 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:12.446265 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9c311d-484b-43dd-95da-537edf975138 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:12.446824 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9c311d-484b-43dd-95da-537edf975138 None admin] PATH_INFO: `/v3/domains` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:12.447746 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9c311d-484b-43dd-95da-537edf975138 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:47:12.448371 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9c311d-484b-43dd-95da-537edf975138 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:12.449975 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9c311d-484b-43dd-95da-537edf975138 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:12.471639 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 364/697] 10.222.0.46 () {56 vars in 1109 bytes} [Mon Oct 10 11:47:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 330 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:47:12.480800 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcee567b-179e-41fb-9a83-79f61892d226 None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:12.648667 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:12.757439 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcee567b-179e-41fb-9a83-79f61892d226 None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:12.823087 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c68f80-0431-4bff-ad5f-1b8114fd3ea4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:12.824632 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c68f80-0431-4bff-ad5f-1b8114fd3ea4 None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:12.825455 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c68f80-0431-4bff-ad5f-1b8114fd3ea4 None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:12.826235 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c68f80-0431-4bff-ad5f-1b8114fd3ea4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:12.839709 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c68f80-0431-4bff-ad5f-1b8114fd3ea4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:12.843167 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c68f80-0431-4bff-ad5f-1b8114fd3ea4 None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:12.867858 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application [None req-66c68f80-0431-4bff-ad5f-1b8114fd3ea4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 10 11:47:12.867858 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:47:12.867858 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:47:12.867858 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:47:12.867858 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:47:12.867858 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:47:12.867858 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:47:12.867858 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:47:12.867858 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:47:12.867858 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:47:12.867858 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:47:12.867858 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:47:12.867858 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 10 11:47:12.867858 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 10 11:47:12.867858 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 10 11:47:12.869520 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 10 11:47:12.869520 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:47:12.869520 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:47:12.869520 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:47:12.869520 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:47:12.869520 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:47:12.869520 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:47:12.869520 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:47:12.869520 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application with Lock( Oct 10 11:47:12.869520 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:47:12.869520 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:47:12.869520 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:47:12.869520 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:47:12.869520 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:47:12.869520 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:47:12.869520 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:47:12.869520 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:47:12.871461 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 10 11:47:12.871461 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 10 11:47:12.871461 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 10 11:47:12.871461 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 10 11:47:12.871461 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 10 11:47:12.871461 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 10 11:47:12.871461 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 10 11:47:12.871461 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Oct 10 11:47:12.876615 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 334/698] 10.222.0.46 () {56 vars in 1111 bytes} [Mon Oct 10 11:47:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 397 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:47:12.883536 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9c311d-484b-43dd-95da-537edf975138 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:12.918941 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:13.087032 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e2f8e3-f0d4-46e2-91c7-4435c837af03 None None] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:13.088689 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e2f8e3-f0d4-46e2-91c7-4435c837af03 None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:13.089877 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e2f8e3-f0d4-46e2-91c7-4435c837af03 None None] PATH_INFO: `/` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:13.095887 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 335/699] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 10 11:47:13 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:47:13.104902 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9c311d-484b-43dd-95da-537edf975138 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:13.108145 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1f5c6c-7e74-4f5e-a43c-abe3827ce230 None None] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:13.109080 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1f5c6c-7e74-4f5e-a43c-abe3827ce230 None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:13.109861 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1f5c6c-7e74-4f5e-a43c-abe3827ce230 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:13.152413 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.auth.core [None req-1b1f5c6c-7e74-4f5e-a43c-abe3827ce230 None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:47:13.155582 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-1b1f5c6c-7e74-4f5e-a43c-abe3827ce230 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:47:13.183090 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7845b7-8b40-4535-a069-22f790b55a00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:13.184002 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7845b7-8b40-4535-a069-22f790b55a00 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:13.185282 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7845b7-8b40-4535-a069-22f790b55a00 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:13.185573 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7845b7-8b40-4535-a069-22f790b55a00 None admin] PATH_INFO: `/v3/projects` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:13.186277 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7845b7-8b40-4535-a069-22f790b55a00 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:47:13.186580 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7845b7-8b40-4535-a069-22f790b55a00 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:13.188802 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7845b7-8b40-4535-a069-22f790b55a00 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:13.213220 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 365/700] 10.222.0.46 () {56 vars in 1148 bytes} [Mon Oct 10 11:47:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 331 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:47:13.216085 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fd33e4-b685-4f92-9bf7-cca26731b008 None None] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:13.216950 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fd33e4-b685-4f92-9bf7-cca26731b008 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:13.218021 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fd33e4-b685-4f92-9bf7-cca26731b008 None None] PATH_INFO: `/` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:13.221915 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 366/701] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 10 11:47:13 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:47:13.223721 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60fd33e4-b685-4f92-9bf7-cca26731b008 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:13.257149 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 336/702] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 10 11:47:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 10 11:47:13.261812 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbccc7e-7ba0-4873-b49b-82c1e2156c79 None None] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:13.261812 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbccc7e-7ba0-4873-b49b-82c1e2156c79 None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:13.261812 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbccc7e-7ba0-4873-b49b-82c1e2156c79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:13.279559 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.auth.core [None req-fcbccc7e-7ba0-4873-b49b-82c1e2156c79 None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:47:13.280881 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-fcbccc7e-7ba0-4873-b49b-82c1e2156c79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:47:13.391530 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:13.433570 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 337/703] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 10 11:47:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 10 11:47:13.433995 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:13.476557 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9c0de2-d40a-4297-9815-4cf3f895406e None None] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:13.476557 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9c0de2-d40a-4297-9815-4cf3f895406e None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:13.476557 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9c0de2-d40a-4297-9815-4cf3f895406e None None] PATH_INFO: `/` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:13.476557 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 338/704] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 10 11:47:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 10 11:47:13.482508 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60fd33e4-b685-4f92-9bf7-cca26731b008 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:13.482791 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadb0976-0f5d-4c3d-af15-18f235573401 None None] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:13.482791 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadb0976-0f5d-4c3d-af15-18f235573401 None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:13.482791 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadb0976-0f5d-4c3d-af15-18f235573401 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:13.520609 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.auth.core [None req-fadb0976-0f5d-4c3d-af15-18f235573401 None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:47:13.521830 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-fadb0976-0f5d-4c3d-af15-18f235573401 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:47:13.565106 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fe28f5-0a44-41c5-966e-629bc736aab4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:13.565727 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fe28f5-0a44-41c5-966e-629bc736aab4 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:13.566001 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fe28f5-0a44-41c5-966e-629bc736aab4 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:13.566269 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fe28f5-0a44-41c5-966e-629bc736aab4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:13.567029 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0fe28f5-0a44-41c5-966e-629bc736aab4 None admin] RBAC: Adding query filter params (role.id=3c35be38787740f4834c728a2ce73b6b, scope.project.id=4ad3772f7b6a47c28b7ba37a9316a3c1, user.id=15ef3d53ea484237adb0b3d64fdc5480) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:47:13.567334 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0fe28f5-0a44-41c5-966e-629bc736aab4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:13.568769 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0fe28f5-0a44-41c5-966e-629bc736aab4 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:13.593342 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 367/705] 10.222.0.46 () {56 vars in 1374 bytes} [Mon Oct 10 11:47:13 2022] GET /identity/v3/role_assignments?user.id=15ef3d53ea484237adb0b3d64fdc5480&scope.project.id=4ad3772f7b6a47c28b7ba37a9316a3c1&role.id=3c35be38787740f4834c728a2ce73b6b => generated 608 bytes in 370 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:47:13.659540 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 339/706] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 10 11:47:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 10 11:47:13.821404 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fe28f5-0a44-41c5-966e-629bc736aab4 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:13.892355 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadb0976-0f5d-4c3d-af15-18f235573401 None None] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:13.990607 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fe28f5-0a44-41c5-966e-629bc736aab4 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:14.033717 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadb0976-0f5d-4c3d-af15-18f235573401 None None] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:14.039592 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89ce6fd-4ef7-49b7-81b4-973077665f35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:14.040247 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89ce6fd-4ef7-49b7-81b4-973077665f35 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:14.040516 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89ce6fd-4ef7-49b7-81b4-973077665f35 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:14.040790 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89ce6fd-4ef7-49b7-81b4-973077665f35 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:14.046903 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:14.054250 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:14.057314 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89ce6fd-4ef7-49b7-81b4-973077665f35 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:14.059629 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89ce6fd-4ef7-49b7-81b4-973077665f35 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:14.073879 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32841bf0-eb20-4e9b-a7df-ac651830d1ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:14.074538 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32841bf0-eb20-4e9b-a7df-ac651830d1ef None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:14.074847 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32841bf0-eb20-4e9b-a7df-ac651830d1ef None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:14.075115 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32841bf0-eb20-4e9b-a7df-ac651830d1ef None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:14.079838 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application [None req-b89ce6fd-4ef7-49b7-81b4-973077665f35 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:47:14.079838 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:47:14.079838 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:47:14.079838 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:47:14.079838 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:47:14.079838 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:47:14.079838 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:47:14.079838 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:47:14.079838 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:47:14.079838 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:47:14.079838 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:47:14.079838 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:47:14.079838 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 10 11:47:14.079838 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 10 11:47:14.079838 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 10 11:47:14.079838 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 10 11:47:14.083089 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:47:14.083089 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:47:14.083089 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:47:14.083089 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:47:14.083089 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:47:14.083089 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:47:14.083089 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:47:14.083089 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application with Lock( Oct 10 11:47:14.083089 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:47:14.083089 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:47:14.083089 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:47:14.083089 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:47:14.083089 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:47:14.083089 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:47:14.083089 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:47:14.083089 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:47:14.083089 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 10 11:47:14.085498 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 10 11:47:14.085498 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:47:14.085498 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Oct 10 11:47:14.085498 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 368/707] 10.222.0.46 () {56 vars in 1108 bytes} [Mon Oct 10 11:47:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:47:14.090365 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89ce6fd-4ef7-49b7-81b4-973077665f35 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:14.094547 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32841bf0-eb20-4e9b-a7df-ac651830d1ef None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 10 11:47:14.101027 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application [None req-32841bf0-eb20-4e9b-a7df-ac651830d1ef None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 10 11:47:14.101027 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:47:14.101027 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:47:14.101027 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:47:14.101027 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:47:14.101027 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:47:14.101027 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:47:14.101027 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:47:14.101027 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:47:14.101027 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:47:14.101027 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:47:14.101027 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:47:14.101027 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 10 11:47:14.101027 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 10 11:47:14.101027 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 10 11:47:14.101027 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 10 11:47:14.103028 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 10 11:47:14.103028 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 10 11:47:14.103028 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 10 11:47:14.103028 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 10 11:47:14.103028 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:47:14.103028 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:47:14.103028 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:47:14.103028 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:47:14.103028 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:47:14.103028 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:47:14.103028 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:47:14.103028 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application with Lock( Oct 10 11:47:14.103028 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:47:14.103028 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:47:14.103028 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:47:14.103028 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:47:14.104849 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:47:14.104849 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:47:14.104849 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:47:14.104849 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:47:14.104849 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 10 11:47:14.104849 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 10 11:47:14.104849 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 10 11:47:14.104849 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 10 11:47:14.104849 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 10 11:47:14.104849 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 10 11:47:14.104849 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 10 11:47:14.104849 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Oct 10 11:47:14.106495 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 340/708] 10.222.0.46 () {56 vars in 1096 bytes} [Mon Oct 10 11:47:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:47:14.113207 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32841bf0-eb20-4e9b-a7df-ac651830d1ef None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:14.269800 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89ce6fd-4ef7-49b7-81b4-973077665f35 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:14.302528 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32841bf0-eb20-4e9b-a7df-ac651830d1ef None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:14.308688 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25933f2b-8404-4cc6-bdf1-53577de3b7cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:14.309126 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25933f2b-8404-4cc6-bdf1-53577de3b7cb None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:14.309307 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25933f2b-8404-4cc6-bdf1-53577de3b7cb None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:14.309637 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25933f2b-8404-4cc6-bdf1-53577de3b7cb None admin] PATH_INFO: `/v3/domains` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:14.310825 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25933f2b-8404-4cc6-bdf1-53577de3b7cb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:47:14.311130 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25933f2b-8404-4cc6-bdf1-53577de3b7cb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:14.312508 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25933f2b-8404-4cc6-bdf1-53577de3b7cb None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:14.327389 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 369/709] 10.222.0.46 () {56 vars in 1109 bytes} [Mon Oct 10 11:47:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:47:14.328453 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25933f2b-8404-4cc6-bdf1-53577de3b7cb None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:14.372944 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c9ecaf-e4ef-4080-b9e6-10ea89c6473c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:14.374501 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c9ecaf-e4ef-4080-b9e6-10ea89c6473c None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:14.375365 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c9ecaf-e4ef-4080-b9e6-10ea89c6473c None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:14.376193 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c9ecaf-e4ef-4080-b9e6-10ea89c6473c None admin] PATH_INFO: `/v3/roles` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:14.377704 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c9ecaf-e4ef-4080-b9e6-10ea89c6473c None admin] RBAC: Adding query filter params (name=admin) {{(pid=81676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:47:14.378567 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c9ecaf-e4ef-4080-b9e6-10ea89c6473c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:14.381541 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c9ecaf-e4ef-4080-b9e6-10ea89c6473c None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:14.391103 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:14.399593 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 341/710] 10.222.0.46 () {56 vars in 1099 bytes} [Mon Oct 10 11:47:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:47:14.401529 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c9ecaf-e4ef-4080-b9e6-10ea89c6473c None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:14.447904 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25933f2b-8404-4cc6-bdf1-53577de3b7cb None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:14.468467 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:14.490119 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf16a4fe-ef63-4de2-b009-f3fb951fceda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:14.491062 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf16a4fe-ef63-4de2-b009-f3fb951fceda None admin] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:14.491488 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf16a4fe-ef63-4de2-b009-f3fb951fceda None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:14.491903 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf16a4fe-ef63-4de2-b009-f3fb951fceda None admin] PATH_INFO: `/v3/services` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:14.492707 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf16a4fe-ef63-4de2-b009-f3fb951fceda None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:14.494313 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf16a4fe-ef63-4de2-b009-f3fb951fceda None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:14.590562 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 370/711] 10.222.0.46 () {60 vars in 1141 bytes} [Mon Oct 10 11:47:14 2022] POST /identity/v3/services => generated 251 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 11:47:14.593585 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf16a4fe-ef63-4de2-b009-f3fb951fceda None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:14.612213 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c9ecaf-e4ef-4080-b9e6-10ea89c6473c None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:14.705584 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc2f60e-7baa-4d39-a9b8-901f3db0cc93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:14.706585 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc2f60e-7baa-4d39-a9b8-901f3db0cc93 None admin] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:14.706585 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc2f60e-7baa-4d39-a9b8-901f3db0cc93 None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:14.706585 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc2f60e-7baa-4d39-a9b8-901f3db0cc93 None admin] PATH_INFO: `/v3/users` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:14.709564 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc2f60e-7baa-4d39-a9b8-901f3db0cc93 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:14.715561 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc2f60e-7baa-4d39-a9b8-901f3db0cc93 None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:14.843369 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 342/712] 10.222.0.46 () {60 vars in 1132 bytes} [Mon Oct 10 11:47:14 2022] POST /identity/v3/users => generated 254 bytes in 442 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 11:47:14.848545 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf16a4fe-ef63-4de2-b009-f3fb951fceda None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:14.875127 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:14.911999 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a39570-21fc-47d6-8839-ec1bdcb3e71a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:14.913050 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a39570-21fc-47d6-8839-ec1bdcb3e71a None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:14.913879 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a39570-21fc-47d6-8839-ec1bdcb3e71a None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:14.914073 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a39570-21fc-47d6-8839-ec1bdcb3e71a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:14.931800 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a39570-21fc-47d6-8839-ec1bdcb3e71a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:14.934195 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a39570-21fc-47d6-8839-ec1bdcb3e71a None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:14.958888 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application [None req-79a39570-21fc-47d6-8839-ec1bdcb3e71a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:47:14.958888 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:47:14.958888 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:47:14.958888 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:47:14.958888 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:47:14.958888 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:47:14.958888 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:47:14.958888 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:47:14.958888 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:47:14.958888 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:47:14.958888 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:47:14.958888 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:47:14.958888 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 10 11:47:14.958888 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 10 11:47:14.958888 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 10 11:47:14.958888 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 10 11:47:14.960539 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:47:14.960539 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:47:14.960539 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:47:14.960539 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:47:14.960539 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:47:14.960539 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:47:14.960539 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:47:14.960539 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application with Lock( Oct 10 11:47:14.960539 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:47:14.960539 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:47:14.960539 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:47:14.960539 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:47:14.960539 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:47:14.960539 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:47:14.960539 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:47:14.960539 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:47:14.960539 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 10 11:47:14.962128 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 10 11:47:14.962128 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:47:14.962128 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Oct 10 11:47:14.962128 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 371/713] 10.222.0.46 () {56 vars in 1108 bytes} [Mon Oct 10 11:47:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 368 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:47:14.973339 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc2f60e-7baa-4d39-a9b8-901f3db0cc93 None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:15.312366 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:15.342917 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc2f60e-7baa-4d39-a9b8-901f3db0cc93 None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:15.411588 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fe27ca-b1e1-480d-80f4-64739eafbca4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:15.412226 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fe27ca-b1e1-480d-80f4-64739eafbca4 None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:15.412511 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fe27ca-b1e1-480d-80f4-64739eafbca4 None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:15.412768 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fe27ca-b1e1-480d-80f4-64739eafbca4 None admin] PATH_INFO: `/v3/domains` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:15.413428 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8fe27ca-b1e1-480d-80f4-64739eafbca4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:47:15.413795 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8fe27ca-b1e1-480d-80f4-64739eafbca4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:15.415260 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8fe27ca-b1e1-480d-80f4-64739eafbca4 None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:15.437883 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 343/714] 10.222.0.46 () {56 vars in 1109 bytes} [Mon Oct 10 11:47:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 466 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:47:15.447531 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a39570-21fc-47d6-8839-ec1bdcb3e71a None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:15.642928 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a39570-21fc-47d6-8839-ec1bdcb3e71a None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:15.706339 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76abedc4-42d6-4590-a10b-cd9649d643c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:15.707415 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76abedc4-42d6-4590-a10b-cd9649d643c7 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:15.707891 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76abedc4-42d6-4590-a10b-cd9649d643c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:15.708315 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76abedc4-42d6-4590-a10b-cd9649d643c7 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:15.721000 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76abedc4-42d6-4590-a10b-cd9649d643c7 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:15.723188 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76abedc4-42d6-4590-a10b-cd9649d643c7 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:15.745695 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application [None req-76abedc4-42d6-4590-a10b-cd9649d643c7 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 10 11:47:15.745695 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:47:15.745695 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:47:15.745695 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:47:15.745695 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:47:15.745695 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:47:15.745695 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:47:15.745695 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:47:15.745695 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:47:15.745695 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:47:15.745695 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:47:15.745695 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:47:15.745695 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 10 11:47:15.745695 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 10 11:47:15.745695 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 10 11:47:15.745695 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 10 11:47:15.747624 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:47:15.747624 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:47:15.747624 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 10 11:47:15.747624 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 10 11:47:15.747624 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 10 11:47:15.747624 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 10 11:47:15.747624 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:47:15.747624 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:47:15.747624 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:47:15.747624 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:47:15.747624 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:47:15.747624 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application with Lock( Oct 10 11:47:15.747624 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:47:15.747624 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:47:15.747624 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:47:15.747624 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:47:15.747624 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:47:15.747624 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:47:15.749482 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:47:15.749482 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:47:15.749482 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 10 11:47:15.749482 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 10 11:47:15.749482 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 10 11:47:15.749482 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 10 11:47:15.749482 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 10 11:47:15.749482 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 10 11:47:15.749482 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 10 11:47:15.749482 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Oct 10 11:47:15.749482 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 372/715] 10.222.0.46 () {56 vars in 1093 bytes} [Mon Oct 10 11:47:15 2022] GET /identity/v3/users/nova => generated 82 bytes in 301 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:47:15.757747 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fe27ca-b1e1-480d-80f4-64739eafbca4 None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:15.760106 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe2d57f-1ae1-42b4-aaff-63317c8d4c1b None None] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:15.760859 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe2d57f-1ae1-42b4-aaff-63317c8d4c1b None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:15.761615 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe2d57f-1ae1-42b4-aaff-63317c8d4c1b None None] PATH_INFO: `/` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:15.764979 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 373/716] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 10 11:47:15 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 10 11:47:15.786465 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e973775-2f29-4459-a6e0-3c629528f282 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:15.787469 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e973775-2f29-4459-a6e0-3c629528f282 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:15.788262 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e973775-2f29-4459-a6e0-3c629528f282 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:15.845496 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-3e973775-2f29-4459-a6e0-3c629528f282 None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:47:15.854715 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-3e973775-2f29-4459-a6e0-3c629528f282 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:47:15.943298 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:15.962334 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fe27ca-b1e1-480d-80f4-64739eafbca4 None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:16.031369 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85da667-d567-4bc6-8861-674b19dc2eaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:16.032362 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85da667-d567-4bc6-8861-674b19dc2eaa None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:16.032828 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85da667-d567-4bc6-8861-674b19dc2eaa None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:16.033266 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85da667-d567-4bc6-8861-674b19dc2eaa None admin] PATH_INFO: `/v3/users` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:16.034583 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85da667-d567-4bc6-8861-674b19dc2eaa None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:47:16.035139 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85da667-d567-4bc6-8861-674b19dc2eaa None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:16.037720 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85da667-d567-4bc6-8861-674b19dc2eaa None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:16.093411 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 344/717] 10.222.0.46 () {56 vars in 1133 bytes} [Mon Oct 10 11:47:15 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 337 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:47:16.096959 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85da667-d567-4bc6-8861-674b19dc2eaa None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:16.104755 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:16.120739 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 374/718] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 10 11:47:15 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 336 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 10 11:47:16.301926 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85da667-d567-4bc6-8861-674b19dc2eaa None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:16.352970 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd05350-1225-434a-9581-51a6839df010 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:16.353604 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd05350-1225-434a-9581-51a6839df010 None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:16.353879 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd05350-1225-434a-9581-51a6839df010 None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:16.354148 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd05350-1225-434a-9581-51a6839df010 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:16.449753 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd05350-1225-434a-9581-51a6839df010 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:16.451929 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd05350-1225-434a-9581-51a6839df010 None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:16.454158 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:16.569890 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 345/719] 10.222.0.46 () {58 vars in 1282 bytes} [Mon Oct 10 11:47:16 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 474 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 10 11:47:16.575088 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e973775-2f29-4459-a6e0-3c629528f282 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:16.796504 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e973775-2f29-4459-a6e0-3c629528f282 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:16.852053 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd05350-1225-434a-9581-51a6839df010 None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:16.861760 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82551d80-a204-40b7-bfa0-484767f043f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:16.862919 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82551d80-a204-40b7-bfa0-484767f043f3 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:16.863491 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82551d80-a204-40b7-bfa0-484767f043f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:16.863971 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82551d80-a204-40b7-bfa0-484767f043f3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:16.885611 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82551d80-a204-40b7-bfa0-484767f043f3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:16.889265 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82551d80-a204-40b7-bfa0-484767f043f3 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:16.918943 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application [None req-82551d80-a204-40b7-bfa0-484767f043f3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:47:16.918943 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:47:16.918943 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:47:16.918943 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:47:16.918943 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:47:16.918943 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:47:16.918943 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:47:16.918943 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:47:16.918943 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:47:16.918943 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:47:16.918943 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:47:16.918943 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:47:16.918943 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 10 11:47:16.918943 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 10 11:47:16.918943 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 10 11:47:16.918943 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 10 11:47:16.922434 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:47:16.922434 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:47:16.922434 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:47:16.922434 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:47:16.922434 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:47:16.922434 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:47:16.922434 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:47:16.922434 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application with Lock( Oct 10 11:47:16.922434 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:47:16.922434 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:47:16.922434 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:47:16.922434 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:47:16.922434 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:47:16.922434 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:47:16.922434 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:47:16.922434 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:47:16.922434 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 10 11:47:16.924601 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 10 11:47:16.924601 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:47:16.924601 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Oct 10 11:47:16.924601 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 375/720] 10.222.0.46 () {56 vars in 1108 bytes} [Mon Oct 10 11:47:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 347 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:47:16.927783 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82551d80-a204-40b7-bfa0-484767f043f3 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:16.944453 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:17.117521 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82551d80-a204-40b7-bfa0-484767f043f3 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:17.174473 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd05350-1225-434a-9581-51a6839df010 None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:17.182581 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:17.232056 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05923b8-b8ce-4aef-b643-13e6bc2d0ad2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:17.232985 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05923b8-b8ce-4aef-b643-13e6bc2d0ad2 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:17.233435 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05923b8-b8ce-4aef-b643-13e6bc2d0ad2 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:17.233895 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05923b8-b8ce-4aef-b643-13e6bc2d0ad2 None admin] PATH_INFO: `/v3/domains` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:17.235096 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05923b8-b8ce-4aef-b643-13e6bc2d0ad2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:47:17.235605 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05923b8-b8ce-4aef-b643-13e6bc2d0ad2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:17.243521 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05923b8-b8ce-4aef-b643-13e6bc2d0ad2 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:17.281128 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 376/721] 10.222.0.46 () {56 vars in 1109 bytes} [Mon Oct 10 11:47:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 353 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:47:17.283754 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05923b8-b8ce-4aef-b643-13e6bc2d0ad2 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:17.298640 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa129c8d-3400-47c7-b8d3-a641607eaa03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:17.299705 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa129c8d-3400-47c7-b8d3-a641607eaa03 None admin] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:17.300181 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa129c8d-3400-47c7-b8d3-a641607eaa03 None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:17.300645 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa129c8d-3400-47c7-b8d3-a641607eaa03 None admin] PATH_INFO: `/v3/services` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:17.301794 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa129c8d-3400-47c7-b8d3-a641607eaa03 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:17.304304 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa129c8d-3400-47c7-b8d3-a641607eaa03 None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:17.393704 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 346/722] 10.222.0.46 () {60 vars in 1141 bytes} [Mon Oct 10 11:47:16 2022] POST /identity/v3/services => generated 240 bytes in 542 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 11:47:17.463456 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:17.520194 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05923b8-b8ce-4aef-b643-13e6bc2d0ad2 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:17.587683 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5780a7a7-fe91-4d67-a688-2c15545c6da9 None None] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:17.588176 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5780a7a7-fe91-4d67-a688-2c15545c6da9 None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:17.588657 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5780a7a7-fe91-4d67-a688-2c15545c6da9 None None] PATH_INFO: `/` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:17.596244 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 347/723] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 10 11:47:17 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:47:17.601318 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a2d517-c55b-404f-86ee-88bd47106c47 None None] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:17.601800 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a2d517-c55b-404f-86ee-88bd47106c47 None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:17.602274 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a2d517-c55b-404f-86ee-88bd47106c47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:17.664710 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.auth.core [None req-44a2d517-c55b-404f-86ee-88bd47106c47 None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:47:17.666790 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-44a2d517-c55b-404f-86ee-88bd47106c47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:47:17.701624 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcda2d5e-6d4d-4e68-81d9-a8fa27e4c9a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:17.702339 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcda2d5e-6d4d-4e68-81d9-a8fa27e4c9a9 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:17.702622 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcda2d5e-6d4d-4e68-81d9-a8fa27e4c9a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:17.703389 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcda2d5e-6d4d-4e68-81d9-a8fa27e4c9a9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:17.710859 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcda2d5e-6d4d-4e68-81d9-a8fa27e4c9a9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:17.712348 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcda2d5e-6d4d-4e68-81d9-a8fa27e4c9a9 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:17.725092 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application [None req-bcda2d5e-6d4d-4e68-81d9-a8fa27e4c9a9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 10 11:47:17.725092 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:47:17.725092 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:47:17.725092 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:47:17.725092 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:47:17.725092 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:47:17.725092 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:47:17.725092 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:47:17.725092 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:47:17.725092 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:47:17.725092 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:47:17.725092 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:47:17.725092 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 10 11:47:17.725092 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 10 11:47:17.725092 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 10 11:47:17.726047 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 10 11:47:17.726047 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:47:17.726047 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:47:17.726047 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:47:17.726047 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:47:17.726047 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:47:17.726047 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:47:17.726047 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:47:17.726047 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application with Lock( Oct 10 11:47:17.726047 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:47:17.726047 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:47:17.726047 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:47:17.726047 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:47:17.726047 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:47:17.726047 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:47:17.726047 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:47:17.726047 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:47:17.726944 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 10 11:47:17.726944 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 10 11:47:17.726944 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 10 11:47:17.726944 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 10 11:47:17.726944 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 10 11:47:17.726944 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 10 11:47:17.726944 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 10 11:47:17.726944 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Oct 10 11:47:17.726944 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 377/724] 10.222.0.46 () {56 vars in 1111 bytes} [Mon Oct 10 11:47:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 444 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:47:17.732566 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcda2d5e-6d4d-4e68-81d9-a8fa27e4c9a9 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:17.850694 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcda2d5e-6d4d-4e68-81d9-a8fa27e4c9a9 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:17.881248 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:17.891131 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5275af49-6ea5-4827-96db-cc9ba19dc32f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:17.896271 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5275af49-6ea5-4827-96db-cc9ba19dc32f None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:17.896271 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5275af49-6ea5-4827-96db-cc9ba19dc32f None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:17.896271 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5275af49-6ea5-4827-96db-cc9ba19dc32f None admin] PATH_INFO: `/v3/projects` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:17.896271 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5275af49-6ea5-4827-96db-cc9ba19dc32f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:47:17.896271 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5275af49-6ea5-4827-96db-cc9ba19dc32f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:17.896271 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5275af49-6ea5-4827-96db-cc9ba19dc32f None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:17.910392 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 378/725] 10.222.0.46 () {56 vars in 1148 bytes} [Mon Oct 10 11:47:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 11:47:17.914642 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5275af49-6ea5-4827-96db-cc9ba19dc32f None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:17.924402 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 348/726] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 10 11:47:17 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 325 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 10 11:47:18.026254 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5275af49-6ea5-4827-96db-cc9ba19dc32f None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:18.065118 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f98eae-bcf2-47a1-9491-5a08b328fe61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:18.065749 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f98eae-bcf2-47a1-9491-5a08b328fe61 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:18.066030 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f98eae-bcf2-47a1-9491-5a08b328fe61 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:18.066301 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f98eae-bcf2-47a1-9491-5a08b328fe61 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:18.067073 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f98eae-bcf2-47a1-9491-5a08b328fe61 None admin] RBAC: Adding query filter params (role.id=0d822a0674b6495fac623877a3b38025, scope.project.id=4ad3772f7b6a47c28b7ba37a9316a3c1, user.id=5f130bbb69e546ce99b4e14de81d7e54) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:47:18.067383 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f98eae-bcf2-47a1-9491-5a08b328fe61 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:18.068827 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f98eae-bcf2-47a1-9491-5a08b328fe61 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:18.076022 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 379/727] 10.222.0.46 () {56 vars in 1374 bytes} [Mon Oct 10 11:47:17 2022] GET /identity/v3/role_assignments?user.id=5f130bbb69e546ce99b4e14de81d7e54&scope.project.id=4ad3772f7b6a47c28b7ba37a9316a3c1&role.id=0d822a0674b6495fac623877a3b38025 => generated 259 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 11:47:18.250365 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d96170-6d39-4ad7-bb46-4df451414ff5 None None] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:18.251413 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d96170-6d39-4ad7-bb46-4df451414ff5 None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:18.252224 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d96170-6d39-4ad7-bb46-4df451414ff5 None None] PATH_INFO: `/` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:18.255488 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 349/728] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 10 11:47:18 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:47:18.261072 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c52d907-5864-4826-b855-1df88ee7466e None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:18.261476 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c52d907-5864-4826-b855-1df88ee7466e None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:18.261628 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c52d907-5864-4826-b855-1df88ee7466e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:18.288657 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-3c52d907-5864-4826-b855-1df88ee7466e None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:47:18.289878 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-3c52d907-5864-4826-b855-1df88ee7466e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:47:18.341586 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:18.377248 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 380/729] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 10 11:47:18 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 10 11:47:18.440370 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d96170-6d39-4ad7-bb46-4df451414ff5 None None] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:18.587222 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:18.604350 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d96170-6d39-4ad7-bb46-4df451414ff5 None None] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:18.663389 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b390195-71ba-4532-8ae5-6d81ea0fc402 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:18.664137 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b390195-71ba-4532-8ae5-6d81ea0fc402 None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:18.664532 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b390195-71ba-4532-8ae5-6d81ea0fc402 None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:18.664775 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b390195-71ba-4532-8ae5-6d81ea0fc402 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:18.680398 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b390195-71ba-4532-8ae5-6d81ea0fc402 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 10 11:47:18.686245 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application [None req-7b390195-71ba-4532-8ae5-6d81ea0fc402 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 10 11:47:18.686245 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:47:18.686245 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:47:18.686245 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:47:18.686245 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:47:18.686245 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:47:18.686245 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:47:18.686245 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:47:18.686245 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:47:18.686245 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:47:18.686245 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:47:18.686245 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:47:18.686245 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 10 11:47:18.686245 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 10 11:47:18.686245 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 10 11:47:18.686245 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 10 11:47:18.688088 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 10 11:47:18.688088 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 10 11:47:18.688088 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 10 11:47:18.688088 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 10 11:47:18.688088 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:47:18.688088 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:47:18.688088 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:47:18.688088 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:47:18.688088 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:47:18.688088 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:47:18.688088 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:47:18.688088 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application with Lock( Oct 10 11:47:18.688088 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:47:18.688088 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:47:18.688088 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:47:18.688088 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:47:18.689653 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:47:18.689653 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:47:18.689653 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:47:18.689653 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:47:18.689653 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 10 11:47:18.689653 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 10 11:47:18.689653 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 10 11:47:18.689653 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 10 11:47:18.689653 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 10 11:47:18.689653 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 10 11:47:18.689653 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 10 11:47:18.689653 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Oct 10 11:47:18.689653 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 350/730] 10.222.0.46 () {56 vars in 1096 bytes} [Mon Oct 10 11:47:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:47:18.694729 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c52d907-5864-4826-b855-1df88ee7466e None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:18.816616 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c52d907-5864-4826-b855-1df88ee7466e None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:18.857904 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9fabf9-33c6-4170-8b05-a1af8bae890e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:18.858439 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9fabf9-33c6-4170-8b05-a1af8bae890e None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:18.858741 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9fabf9-33c6-4170-8b05-a1af8bae890e None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:18.859015 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9fabf9-33c6-4170-8b05-a1af8bae890e None admin] PATH_INFO: `/v3/roles` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:18.859718 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9fabf9-33c6-4170-8b05-a1af8bae890e None admin] RBAC: Adding query filter params (name=admin) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:47:18.860020 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9fabf9-33c6-4170-8b05-a1af8bae890e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:18.861431 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9fabf9-33c6-4170-8b05-a1af8bae890e None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:18.872182 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 381/731] 10.222.0.46 () {56 vars in 1099 bytes} [Mon Oct 10 11:47:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:47:18.878113 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b390195-71ba-4532-8ae5-6d81ea0fc402 None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:18.913300 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b637e7-5256-4024-8f54-2fc244b1eff0 None None] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:18.913784 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b637e7-5256-4024-8f54-2fc244b1eff0 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:18.914025 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b637e7-5256-4024-8f54-2fc244b1eff0 None None] PATH_INFO: `/` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:18.915298 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 382/732] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 10 11:47:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 10 11:47:18.923662 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9631b2-0fc2-4636-9b07-83dd5b834d1b None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:18.924082 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9631b2-0fc2-4636-9b07-83dd5b834d1b None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:18.924331 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9631b2-0fc2-4636-9b07-83dd5b834d1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:18.943959 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-2b9631b2-0fc2-4636-9b07-83dd5b834d1b None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:47:18.945884 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-2b9631b2-0fc2-4636-9b07-83dd5b834d1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:47:18.987250 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:19.025676 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 383/733] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 10 11:47:18 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 10 11:47:19.055142 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9631b2-0fc2-4636-9b07-83dd5b834d1b None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:19.154977 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b390195-71ba-4532-8ae5-6d81ea0fc402 None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:19.232907 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9631b2-0fc2-4636-9b07-83dd5b834d1b None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:19.235834 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e804483-59e0-467e-9ef5-5602f28c2d51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:19.236869 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e804483-59e0-467e-9ef5-5602f28c2d51 None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:19.237361 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e804483-59e0-467e-9ef5-5602f28c2d51 None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:19.237839 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e804483-59e0-467e-9ef5-5602f28c2d51 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:19.268470 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e804483-59e0-467e-9ef5-5602f28c2d51 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:19.272392 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e804483-59e0-467e-9ef5-5602f28c2d51 None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:19.298961 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application [None req-9e804483-59e0-467e-9ef5-5602f28c2d51 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:47:19.298961 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:47:19.298961 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:47:19.298961 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:47:19.298961 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:47:19.298961 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:47:19.298961 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:47:19.298961 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:47:19.298961 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:47:19.298961 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:47:19.298961 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:47:19.298961 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:47:19.298961 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 10 11:47:19.298961 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 10 11:47:19.298961 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 10 11:47:19.298961 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 10 11:47:19.299948 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:47:19.299948 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:47:19.299948 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:47:19.299948 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:47:19.299948 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:47:19.299948 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:47:19.299948 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:47:19.299948 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application with Lock( Oct 10 11:47:19.299948 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:47:19.299948 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:47:19.299948 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:47:19.299948 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:47:19.299948 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:47:19.299948 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:47:19.299948 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:47:19.299948 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:47:19.299948 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 10 11:47:19.300927 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 10 11:47:19.300927 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:47:19.300927 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Oct 10 11:47:19.301959 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d451377-93cd-4fe9-bfb1-cc1498e01870 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:19.302840 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d451377-93cd-4fe9-bfb1-cc1498e01870 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:19.303277 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d451377-93cd-4fe9-bfb1-cc1498e01870 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:19.303696 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d451377-93cd-4fe9-bfb1-cc1498e01870 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:19.304445 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d451377-93cd-4fe9-bfb1-cc1498e01870 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:19.306018 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d451377-93cd-4fe9-bfb1-cc1498e01870 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:19.307799 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 351/734] 10.222.0.46 () {56 vars in 1108 bytes} [Mon Oct 10 11:47:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 430 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:47:19.316186 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e804483-59e0-467e-9ef5-5602f28c2d51 None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:19.324433 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:19.334512 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application [None req-4d451377-93cd-4fe9-bfb1-cc1498e01870 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 10 11:47:19.334512 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:47:19.334512 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 10 11:47:19.334512 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 10 11:47:19.334512 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 10 11:47:19.334512 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 10 11:47:19.334512 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 10 11:47:19.334512 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 10 11:47:19.334512 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 10 11:47:19.334512 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Oct 10 11:47:19.334512 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 10 11:47:19.334512 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Oct 10 11:47:19.334512 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:47:19.334512 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:47:19.334512 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:47:19.334512 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:47:19.334512 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:47:19.346259 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:47:19.346259 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:47:19.346259 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:47:19.346259 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:47:19.346259 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:47:19.346259 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:47:19.346259 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 10 11:47:19.346259 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 10 11:47:19.346259 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 10 11:47:19.346259 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 10 11:47:19.346259 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:47:19.346259 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:47:19.346259 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:47:19.346259 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:47:19.346259 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:47:19.346259 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:47:19.347895 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:47:19.347895 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application with Lock( Oct 10 11:47:19.347895 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:47:19.347895 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:47:19.347895 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:47:19.347895 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:47:19.347895 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:47:19.347895 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:47:19.347895 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:47:19.347895 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:47:19.347895 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 10 11:47:19.347895 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 10 11:47:19.347895 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 10 11:47:19.347895 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Oct 10 11:47:19.347895 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 384/735] 10.222.0.46 () {56 vars in 1129 bytes} [Mon Oct 10 11:47:19 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:47:19.352148 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d451377-93cd-4fe9-bfb1-cc1498e01870 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:19.396510 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:19.504895 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d451377-93cd-4fe9-bfb1-cc1498e01870 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:19.535756 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e804483-59e0-467e-9ef5-5602f28c2d51 None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:19.583843 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603ed03d-074a-45b9-b45e-44037d9527fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:19.584499 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603ed03d-074a-45b9-b45e-44037d9527fe None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:19.584778 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603ed03d-074a-45b9-b45e-44037d9527fe None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:19.585050 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603ed03d-074a-45b9-b45e-44037d9527fe None admin] PATH_INFO: `/v3/services` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:19.585871 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603ed03d-074a-45b9-b45e-44037d9527fe None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:47:19.586364 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603ed03d-074a-45b9-b45e-44037d9527fe None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:19.588047 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603ed03d-074a-45b9-b45e-44037d9527fe None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:19.601347 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 385/736] 10.222.0.46 () {56 vars in 1124 bytes} [Mon Oct 10 11:47:19 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:47:19.610778 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603ed03d-074a-45b9-b45e-44037d9527fe None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:19.618373 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4e2540-19bc-46be-94ff-a0f89a795e6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:19.619576 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4e2540-19bc-46be-94ff-a0f89a795e6d None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:19.620615 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4e2540-19bc-46be-94ff-a0f89a795e6d None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:19.621337 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4e2540-19bc-46be-94ff-a0f89a795e6d None admin] PATH_INFO: `/v3/domains` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:19.623537 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4e2540-19bc-46be-94ff-a0f89a795e6d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:47:19.624376 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4e2540-19bc-46be-94ff-a0f89a795e6d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:19.627423 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4e2540-19bc-46be-94ff-a0f89a795e6d None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:19.655050 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 352/737] 10.222.0.46 () {56 vars in 1109 bytes} [Mon Oct 10 11:47:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 339 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 11:47:19.657229 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4e2540-19bc-46be-94ff-a0f89a795e6d None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:19.784236 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:19.799682 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:19.813073 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603ed03d-074a-45b9-b45e-44037d9527fe None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:19.878571 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4e2540-19bc-46be-94ff-a0f89a795e6d None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:19.879829 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09979366-5e6a-4965-9495-5565a5bcbe6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:19.881528 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09979366-5e6a-4965-9495-5565a5bcbe6a None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:19.882208 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09979366-5e6a-4965-9495-5565a5bcbe6a None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:19.882898 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09979366-5e6a-4965-9495-5565a5bcbe6a None admin] PATH_INFO: `/v3/services` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:19.884282 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09979366-5e6a-4965-9495-5565a5bcbe6a None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:47:19.885125 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09979366-5e6a-4965-9495-5565a5bcbe6a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:19.887981 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09979366-5e6a-4965-9495-5565a5bcbe6a None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:19.903027 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 386/738] 10.222.0.46 () {56 vars in 1124 bytes} [Mon Oct 10 11:47:19 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 11:47:19.905584 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09979366-5e6a-4965-9495-5565a5bcbe6a None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:19.955361 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c8cf23-225b-4a53-b420-e3f19c0bad9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:19.956336 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c8cf23-225b-4a53-b420-e3f19c0bad9b None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:19.957126 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c8cf23-225b-4a53-b420-e3f19c0bad9b None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:19.957541 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c8cf23-225b-4a53-b420-e3f19c0bad9b None admin] PATH_INFO: `/v3/roles/service` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:19.989214 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c8cf23-225b-4a53-b420-e3f19c0bad9b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 10 11:47:20.002000 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application [None req-e4c8cf23-225b-4a53-b420-e3f19c0bad9b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 10 11:47:20.002000 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:47:20.002000 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:47:20.002000 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:47:20.002000 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:47:20.002000 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:47:20.002000 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:47:20.002000 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:47:20.002000 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:47:20.002000 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:47:20.002000 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:47:20.002000 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:47:20.002000 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 10 11:47:20.002000 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 10 11:47:20.002000 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 10 11:47:20.002000 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 10 11:47:20.003831 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 10 11:47:20.003831 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 10 11:47:20.003831 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 10 11:47:20.003831 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 10 11:47:20.003831 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:47:20.003831 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:47:20.003831 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:47:20.003831 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:47:20.003831 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:47:20.003831 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:47:20.003831 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:47:20.003831 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application with Lock( Oct 10 11:47:20.003831 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:47:20.003831 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:47:20.003831 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:47:20.003831 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:47:20.005528 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:47:20.005528 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:47:20.005528 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:47:20.005528 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:47:20.005528 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 10 11:47:20.005528 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 10 11:47:20.005528 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 10 11:47:20.005528 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 10 11:47:20.005528 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 10 11:47:20.005528 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 10 11:47:20.005528 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 10 11:47:20.005528 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Oct 10 11:47:20.011408 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 353/739] 10.222.0.46 () {56 vars in 1102 bytes} [Mon Oct 10 11:47:19 2022] GET /identity/v3/roles/service => generated 85 bytes in 349 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:47:20.013088 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c8cf23-225b-4a53-b420-e3f19c0bad9b None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:20.122611 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09979366-5e6a-4965-9495-5565a5bcbe6a None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:20.188094 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10a7762-4066-4017-be28-80a107dfee1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:20.189638 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10a7762-4066-4017-be28-80a107dfee1f None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:20.190394 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10a7762-4066-4017-be28-80a107dfee1f None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:20.191190 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10a7762-4066-4017-be28-80a107dfee1f None admin] PATH_INFO: `/v3/users/swift` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:20.205184 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10a7762-4066-4017-be28-80a107dfee1f None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:20.208103 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10a7762-4066-4017-be28-80a107dfee1f None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:20.213703 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c8cf23-225b-4a53-b420-e3f19c0bad9b None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:20.230806 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application [None req-f10a7762-4066-4017-be28-80a107dfee1f None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 10 11:47:20.230806 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:47:20.230806 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:47:20.230806 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:47:20.230806 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:47:20.230806 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:47:20.230806 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:47:20.230806 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:47:20.230806 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:47:20.230806 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:47:20.230806 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:47:20.230806 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:47:20.230806 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 10 11:47:20.230806 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 10 11:47:20.230806 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 10 11:47:20.230806 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 10 11:47:20.233044 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:47:20.233044 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:47:20.233044 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 10 11:47:20.233044 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 10 11:47:20.233044 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 10 11:47:20.233044 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 10 11:47:20.233044 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:47:20.233044 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:47:20.233044 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:47:20.233044 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:47:20.233044 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:47:20.233044 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application with Lock( Oct 10 11:47:20.233044 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:47:20.233044 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:47:20.233044 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:47:20.233044 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:47:20.233044 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:47:20.233044 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:47:20.234875 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:47:20.234875 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:47:20.234875 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 10 11:47:20.234875 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 10 11:47:20.234875 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 10 11:47:20.234875 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 10 11:47:20.234875 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 10 11:47:20.234875 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 10 11:47:20.234875 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 10 11:47:20.234875 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Oct 10 11:47:20.239289 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 387/740] 10.222.0.46 () {56 vars in 1096 bytes} [Mon Oct 10 11:47:19 2022] GET /identity/v3/users/swift => generated 83 bytes in 335 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:47:20.241106 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10a7762-4066-4017-be28-80a107dfee1f None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:20.261855 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:20.284523 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6729898a-edb2-4e15-ac9a-21d28b868435 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:20.285248 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6729898a-edb2-4e15-ac9a-21d28b868435 None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:20.292276 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:20.292473 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6729898a-edb2-4e15-ac9a-21d28b868435 None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:20.292473 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6729898a-edb2-4e15-ac9a-21d28b868435 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:20.292473 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6729898a-edb2-4e15-ac9a-21d28b868435 None admin] RBAC: Adding query filter params (interface=public, service_id=c5702a6e30bd462dbc0d6f94532a9815, region_id=RegionOne) {{(pid=81676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:47:20.292473 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6729898a-edb2-4e15-ac9a-21d28b868435 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:20.292473 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6729898a-edb2-4e15-ac9a-21d28b868435 None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:20.306843 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 354/741] 10.222.0.46 () {56 vars in 1251 bytes} [Mon Oct 10 11:47:20 2022] GET /identity/v3/endpoints?service_id=c5702a6e30bd462dbc0d6f94532a9815&interface=public®ion_id=RegionOne => generated 194 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:47:20.308522 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6729898a-edb2-4e15-ac9a-21d28b868435 None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:20.457282 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10a7762-4066-4017-be28-80a107dfee1f None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:20.518074 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6729898a-edb2-4e15-ac9a-21d28b868435 None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:20.548454 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94990f4-04ae-417e-ac9d-820b6d2fcdbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:20.550219 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94990f4-04ae-417e-ac9d-820b6d2fcdbe None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:20.551105 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94990f4-04ae-417e-ac9d-820b6d2fcdbe None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:20.551867 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94990f4-04ae-417e-ac9d-820b6d2fcdbe None admin] PATH_INFO: `/v3/roles` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:20.553593 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94990f4-04ae-417e-ac9d-820b6d2fcdbe None admin] RBAC: Adding query filter params (name=service) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:47:20.554702 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94990f4-04ae-417e-ac9d-820b6d2fcdbe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:20.557825 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94990f4-04ae-417e-ac9d-820b6d2fcdbe None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:20.580160 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 388/742] 10.222.0.46 () {56 vars in 1103 bytes} [Mon Oct 10 11:47:20 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 340 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:47:20.582194 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94990f4-04ae-417e-ac9d-820b6d2fcdbe None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:20.594208 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9b9538-9503-4c7d-998c-1e82a3d1d94f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:20.598934 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9b9538-9503-4c7d-998c-1e82a3d1d94f None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:20.598934 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9b9538-9503-4c7d-998c-1e82a3d1d94f None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:20.598934 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9b9538-9503-4c7d-998c-1e82a3d1d94f None admin] PATH_INFO: `/v3/users` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:20.598934 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9b9538-9503-4c7d-998c-1e82a3d1d94f None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:47:20.598934 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9b9538-9503-4c7d-998c-1e82a3d1d94f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:20.600827 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9b9538-9503-4c7d-998c-1e82a3d1d94f None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:20.638232 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 355/743] 10.222.0.46 () {56 vars in 1135 bytes} [Mon Oct 10 11:47:20 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 331 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:47:20.639959 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9b9538-9503-4c7d-998c-1e82a3d1d94f None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:20.709895 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:20.752539 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:20.787671 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94990f4-04ae-417e-ac9d-820b6d2fcdbe None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:20.797691 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9b9538-9503-4c7d-998c-1e82a3d1d94f None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:20.846508 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a11faf3-3590-4082-85e4-d644a82a2d87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:20.847188 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a11faf3-3590-4082-85e4-d644a82a2d87 None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:20.847610 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a11faf3-3590-4082-85e4-d644a82a2d87 None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:20.847784 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a11faf3-3590-4082-85e4-d644a82a2d87 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:20.857843 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f235c63f-19a1-4a86-929c-a32b5ad1c354 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:20.858932 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f235c63f-19a1-4a86-929c-a32b5ad1c354 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:20.859433 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f235c63f-19a1-4a86-929c-a32b5ad1c354 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:20.859916 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f235c63f-19a1-4a86-929c-a32b5ad1c354 None admin] PATH_INFO: `/v3/services` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:20.861040 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f235c63f-19a1-4a86-929c-a32b5ad1c354 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:20.861323 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a11faf3-3590-4082-85e4-d644a82a2d87 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:20.862831 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a11faf3-3590-4082-85e4-d644a82a2d87 None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:20.863613 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f235c63f-19a1-4a86-929c-a32b5ad1c354 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:20.878720 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 389/744] 10.222.0.46 () {56 vars in 1087 bytes} [Mon Oct 10 11:47:20 2022] GET /identity/v3/services => generated 772 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:47:20.878720 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f235c63f-19a1-4a86-929c-a32b5ad1c354 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:20.895375 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application [None req-1a11faf3-3590-4082-85e4-d644a82a2d87 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:47:20.895375 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:47:20.895375 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:47:20.895375 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:47:20.895375 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:47:20.895375 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:47:20.895375 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:47:20.895375 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:47:20.895375 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:47:20.895375 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:47:20.895375 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:47:20.895375 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:47:20.895375 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 10 11:47:20.895375 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 10 11:47:20.895375 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 10 11:47:20.895375 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 10 11:47:20.897261 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:47:20.897261 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:47:20.897261 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:47:20.897261 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:47:20.897261 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:47:20.897261 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:47:20.897261 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:47:20.897261 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application with Lock( Oct 10 11:47:20.897261 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:47:20.897261 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:47:20.897261 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:47:20.897261 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:47:20.897261 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:47:20.897261 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:47:20.897261 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:47:20.897261 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:47:20.897261 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 10 11:47:20.899838 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 10 11:47:20.899838 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:47:20.899838 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Oct 10 11:47:20.899838 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 356/745] 10.222.0.46 () {56 vars in 1108 bytes} [Mon Oct 10 11:47:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:47:20.901587 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba39e8be-3395-4b3d-bdc8-5d4758a5434b None None] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:20.901587 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba39e8be-3395-4b3d-bdc8-5d4758a5434b None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:20.901587 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba39e8be-3395-4b3d-bdc8-5d4758a5434b None None] PATH_INFO: `/` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:20.910382 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 357/746] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 10 11:47:20 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:47:20.912553 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d964e9-c741-4809-a80d-105ae2556cb3 None None] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:20.912950 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d964e9-c741-4809-a80d-105ae2556cb3 None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:20.913367 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d964e9-c741-4809-a80d-105ae2556cb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:20.943564 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.auth.core [None req-f3d964e9-c741-4809-a80d-105ae2556cb3 None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:47:20.945570 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-f3d964e9-c741-4809-a80d-105ae2556cb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:47:21.074797 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 358/747] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 10 11:47:20 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 10 11:47:21.074797 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d964e9-c741-4809-a80d-105ae2556cb3 None None] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:21.098443 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f235c63f-19a1-4a86-929c-a32b5ad1c354 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:21.115027 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:21.167213 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d878e79-92b1-4447-a350-2d7cd1ac3d82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:21.168437 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d878e79-92b1-4447-a350-2d7cd1ac3d82 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:21.169558 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d878e79-92b1-4447-a350-2d7cd1ac3d82 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:21.170128 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d878e79-92b1-4447-a350-2d7cd1ac3d82 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:21.202321 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:21.275647 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d964e9-c741-4809-a80d-105ae2556cb3 None None] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:21.314908 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d878e79-92b1-4447-a350-2d7cd1ac3d82 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:21.318258 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d878e79-92b1-4447-a350-2d7cd1ac3d82 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:21.350102 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3872e2-97f6-45cd-a7eb-65dc67709a82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:21.351730 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3872e2-97f6-45cd-a7eb-65dc67709a82 None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:21.352356 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3872e2-97f6-45cd-a7eb-65dc67709a82 None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:21.353053 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3872e2-97f6-45cd-a7eb-65dc67709a82 None admin] PATH_INFO: `/v3/domains` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:21.354471 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3872e2-97f6-45cd-a7eb-65dc67709a82 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:47:21.355269 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3872e2-97f6-45cd-a7eb-65dc67709a82 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:21.358327 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3872e2-97f6-45cd-a7eb-65dc67709a82 None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:21.382435 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 359/748] 10.222.0.46 () {56 vars in 1109 bytes} [Mon Oct 10 11:47:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 317 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:47:21.384741 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5293a636-2fde-44ff-bc28-18281a5eff5e None None] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:21.385221 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5293a636-2fde-44ff-bc28-18281a5eff5e None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:21.385700 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5293a636-2fde-44ff-bc28-18281a5eff5e None None] PATH_INFO: `/` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:21.392192 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 360/749] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 10 11:47:21 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:47:21.392192 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5293a636-2fde-44ff-bc28-18281a5eff5e None None] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:21.526605 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 390/750] 10.222.0.46 () {58 vars in 1282 bytes} [Mon Oct 10 11:47:20 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 652 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 10 11:47:21.527810 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d878e79-92b1-4447-a350-2d7cd1ac3d82 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:21.554989 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:21.572805 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5293a636-2fde-44ff-bc28-18281a5eff5e None None] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:21.624642 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:21.637031 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cba665e-0223-493f-9de5-d8d761692dfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:21.638910 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cba665e-0223-493f-9de5-d8d761692dfe None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:21.639065 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cba665e-0223-493f-9de5-d8d761692dfe None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:21.639406 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cba665e-0223-493f-9de5-d8d761692dfe None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:21.657883 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cba665e-0223-493f-9de5-d8d761692dfe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 10 11:47:21.666003 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application [None req-1cba665e-0223-493f-9de5-d8d761692dfe None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 10 11:47:21.666003 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:47:21.666003 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:47:21.666003 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:47:21.666003 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:47:21.666003 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:47:21.666003 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:47:21.666003 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:47:21.666003 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:47:21.666003 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:47:21.666003 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:47:21.666003 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:47:21.666003 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 10 11:47:21.666003 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 10 11:47:21.666003 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 10 11:47:21.666003 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 10 11:47:21.670359 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 10 11:47:21.670359 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 10 11:47:21.670359 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 10 11:47:21.670359 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 10 11:47:21.670359 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:47:21.670359 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:47:21.670359 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:47:21.670359 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:47:21.670359 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:47:21.670359 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:47:21.670359 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:47:21.670359 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application with Lock( Oct 10 11:47:21.670359 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:47:21.670359 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:47:21.670359 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:47:21.670359 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:47:21.671839 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:47:21.671839 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:47:21.671839 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:47:21.671839 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:47:21.671839 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 10 11:47:21.671839 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 10 11:47:21.671839 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 10 11:47:21.671839 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 10 11:47:21.671839 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 10 11:47:21.671839 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 10 11:47:21.671839 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 10 11:47:21.671839 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Oct 10 11:47:21.671839 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 361/751] 10.222.0.46 () {56 vars in 1096 bytes} [Mon Oct 10 11:47:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:47:21.671839 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26e2407-9de3-478a-b16d-6aa8fc0dd970 None None] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:21.674764 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26e2407-9de3-478a-b16d-6aa8fc0dd970 None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:21.674764 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26e2407-9de3-478a-b16d-6aa8fc0dd970 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:21.701479 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.auth.core [None req-f26e2407-9de3-478a-b16d-6aa8fc0dd970 None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:47:21.703444 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-f26e2407-9de3-478a-b16d-6aa8fc0dd970 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:47:21.736636 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d878e79-92b1-4447-a350-2d7cd1ac3d82 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:21.802558 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a046903e-5dc3-406d-9cb6-be077d7a3798 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:21.803634 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a046903e-5dc3-406d-9cb6-be077d7a3798 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:21.803932 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a046903e-5dc3-406d-9cb6-be077d7a3798 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:21.804244 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a046903e-5dc3-406d-9cb6-be077d7a3798 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:21.817969 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a046903e-5dc3-406d-9cb6-be077d7a3798 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:21.820284 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a046903e-5dc3-406d-9cb6-be077d7a3798 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:21.825357 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 362/752] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 10 11:47:21 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 10 11:47:21.826907 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26e2407-9de3-478a-b16d-6aa8fc0dd970 None None] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:21.855791 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application [None req-a046903e-5dc3-406d-9cb6-be077d7a3798 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 10 11:47:21.855791 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:47:21.855791 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:47:21.855791 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:47:21.855791 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:47:21.855791 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:47:21.855791 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:47:21.855791 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:47:21.855791 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:47:21.855791 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:47:21.855791 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:47:21.855791 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:47:21.855791 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 10 11:47:21.855791 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 10 11:47:21.855791 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 10 11:47:21.855791 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 10 11:47:21.858150 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:47:21.858150 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:47:21.858150 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 10 11:47:21.858150 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 10 11:47:21.858150 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 10 11:47:21.858150 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 10 11:47:21.858150 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:47:21.858150 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:47:21.858150 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:47:21.858150 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:47:21.858150 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:47:21.858150 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application with Lock( Oct 10 11:47:21.858150 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:47:21.858150 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:47:21.858150 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:47:21.858150 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:47:21.858150 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:47:21.858150 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:47:21.860647 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:47:21.860647 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:47:21.860647 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 10 11:47:21.860647 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 10 11:47:21.860647 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 10 11:47:21.860647 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 10 11:47:21.860647 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 10 11:47:21.860647 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 10 11:47:21.860647 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 10 11:47:21.860647 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Oct 10 11:47:21.861747 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 391/753] 10.222.0.46 () {56 vars in 1117 bytes} [Mon Oct 10 11:47:21 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 334 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:47:21.863045 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a046903e-5dc3-406d-9cb6-be077d7a3798 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:22.008446 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:22.053815 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26e2407-9de3-478a-b16d-6aa8fc0dd970 None None] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:22.089795 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a046903e-5dc3-406d-9cb6-be077d7a3798 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:22.106419 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:22.129484 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29d9030-2873-4bbd-9f28-3bcbaf06f765 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:22.130528 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29d9030-2873-4bbd-9f28-3bcbaf06f765 None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:22.131031 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29d9030-2873-4bbd-9f28-3bcbaf06f765 None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:22.131523 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29d9030-2873-4bbd-9f28-3bcbaf06f765 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:22.154771 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29d9030-2873-4bbd-9f28-3bcbaf06f765 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:22.157952 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29d9030-2873-4bbd-9f28-3bcbaf06f765 None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:22.172615 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2257de75-3558-4675-b458-df41aa709d24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:22.173534 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2257de75-3558-4675-b458-df41aa709d24 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:22.173947 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2257de75-3558-4675-b458-df41aa709d24 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:22.174345 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2257de75-3558-4675-b458-df41aa709d24 None admin] PATH_INFO: `/v3/roles` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:22.175450 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2257de75-3558-4675-b458-df41aa709d24 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:47:22.175972 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2257de75-3558-4675-b458-df41aa709d24 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:22.179414 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2257de75-3558-4675-b458-df41aa709d24 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:22.191587 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application [None req-e29d9030-2873-4bbd-9f28-3bcbaf06f765 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:47:22.191587 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:47:22.191587 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:47:22.191587 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:47:22.191587 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:47:22.191587 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:47:22.191587 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:47:22.191587 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:47:22.191587 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:47:22.191587 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:47:22.191587 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:47:22.191587 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:47:22.191587 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 10 11:47:22.191587 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 10 11:47:22.191587 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 10 11:47:22.191587 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 10 11:47:22.193411 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:47:22.193411 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:47:22.193411 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:47:22.193411 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:47:22.193411 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:47:22.193411 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:47:22.193411 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:47:22.193411 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application with Lock( Oct 10 11:47:22.193411 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:47:22.193411 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:47:22.193411 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:47:22.193411 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:47:22.193411 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:47:22.193411 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:47:22.193411 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:47:22.193411 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:47:22.193411 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 10 11:47:22.195052 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 10 11:47:22.195052 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:47:22.195052 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Oct 10 11:47:22.198741 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 363/754] 10.222.0.46 () {56 vars in 1108 bytes} [Mon Oct 10 11:47:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 373 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:47:22.201307 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29d9030-2873-4bbd-9f28-3bcbaf06f765 None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:22.203035 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 392/755] 10.222.0.46 () {56 vars in 1099 bytes} [Mon Oct 10 11:47:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 341 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:47:22.204581 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2257de75-3558-4675-b458-df41aa709d24 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:22.378353 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2257de75-3558-4675-b458-df41aa709d24 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:22.403548 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29d9030-2873-4bbd-9f28-3bcbaf06f765 None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:22.426438 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80c747a-199c-44c3-9899-4d0286449d14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:22.427813 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80c747a-199c-44c3-9899-4d0286449d14 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:22.428262 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80c747a-199c-44c3-9899-4d0286449d14 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:22.428659 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80c747a-199c-44c3-9899-4d0286449d14 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:22.447010 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80c747a-199c-44c3-9899-4d0286449d14 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:22.453224 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80c747a-199c-44c3-9899-4d0286449d14 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:22.454390 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:22.465160 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8118f84-162a-4b70-b2bd-42f390aeb12f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:22.466870 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8118f84-162a-4b70-b2bd-42f390aeb12f None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:22.467444 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8118f84-162a-4b70-b2bd-42f390aeb12f None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:22.468033 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8118f84-162a-4b70-b2bd-42f390aeb12f None admin] PATH_INFO: `/v3/users` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:22.469512 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8118f84-162a-4b70-b2bd-42f390aeb12f None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:47:22.470088 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8118f84-162a-4b70-b2bd-42f390aeb12f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:22.472589 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8118f84-162a-4b70-b2bd-42f390aeb12f None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:22.481168 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application [None req-a80c747a-199c-44c3-9899-4d0286449d14 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:47:22.481168 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:47:22.481168 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:47:22.481168 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:47:22.481168 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:47:22.481168 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:47:22.481168 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:47:22.481168 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:47:22.481168 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:47:22.481168 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:47:22.481168 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:47:22.481168 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:47:22.481168 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 10 11:47:22.481168 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 10 11:47:22.481168 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 10 11:47:22.481168 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 10 11:47:22.483405 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:47:22.483405 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:47:22.483405 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:47:22.483405 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:47:22.483405 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:47:22.483405 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:47:22.483405 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:47:22.483405 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application with Lock( Oct 10 11:47:22.483405 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:47:22.483405 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:47:22.483405 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:47:22.483405 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:47:22.483405 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:47:22.483405 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:47:22.483405 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:47:22.483405 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:47:22.483405 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 10 11:47:22.485093 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 10 11:47:22.485093 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:47:22.485093 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Oct 10 11:47:22.490508 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 393/756] 10.222.0.46 () {56 vars in 1108 bytes} [Mon Oct 10 11:47:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:47:22.491597 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80c747a-199c-44c3-9899-4d0286449d14 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:22.514092 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 364/757] 10.222.0.46 () {56 vars in 1149 bytes} [Mon Oct 10 11:47:22 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:47:22.514092 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8118f84-162a-4b70-b2bd-42f390aeb12f None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:22.540430 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:22.675997 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8118f84-162a-4b70-b2bd-42f390aeb12f None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:22.676662 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80c747a-199c-44c3-9899-4d0286449d14 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:22.731792 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46680de7-93b9-46da-a317-17d85510e15e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:22.732752 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46680de7-93b9-46da-a317-17d85510e15e None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:22.733082 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46680de7-93b9-46da-a317-17d85510e15e None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:22.733333 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46680de7-93b9-46da-a317-17d85510e15e None admin] PATH_INFO: `/v3/services/network` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:22.733917 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911ac37a-20d0-49e0-8390-6c022551e706 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:22.734103 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46680de7-93b9-46da-a317-17d85510e15e None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:22.734832 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911ac37a-20d0-49e0-8390-6c022551e706 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:22.735245 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911ac37a-20d0-49e0-8390-6c022551e706 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:22.735401 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46680de7-93b9-46da-a317-17d85510e15e None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:22.735781 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911ac37a-20d0-49e0-8390-6c022551e706 None admin] PATH_INFO: `/v3/domains` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:22.736578 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911ac37a-20d0-49e0-8390-6c022551e706 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:47:22.737012 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911ac37a-20d0-49e0-8390-6c022551e706 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:22.739152 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911ac37a-20d0-49e0-8390-6c022551e706 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:22.745778 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application [None req-46680de7-93b9-46da-a317-17d85510e15e None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Oct 10 11:47:22.745778 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:47:22.745778 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 10 11:47:22.745778 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 10 11:47:22.745778 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 10 11:47:22.745778 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 10 11:47:22.745778 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 10 11:47:22.745778 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 10 11:47:22.745778 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 10 11:47:22.745778 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Oct 10 11:47:22.745778 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 10 11:47:22.745778 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Oct 10 11:47:22.745778 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:47:22.745778 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:47:22.745778 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:47:22.745778 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:47:22.745778 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:47:22.748386 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:47:22.748386 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:47:22.748386 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:47:22.748386 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:47:22.748386 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:47:22.748386 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:47:22.748386 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 10 11:47:22.748386 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 10 11:47:22.748386 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 10 11:47:22.748386 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 10 11:47:22.748386 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:47:22.748386 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:47:22.748386 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:47:22.748386 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:47:22.748386 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:47:22.748386 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:47:22.749562 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:47:22.749562 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application with Lock( Oct 10 11:47:22.749562 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:47:22.749562 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:47:22.749562 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:47:22.749562 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:47:22.749562 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:47:22.749562 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:47:22.749562 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:47:22.749562 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:47:22.749562 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 10 11:47:22.749562 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 10 11:47:22.749562 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 10 11:47:22.749562 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Oct 10 11:47:22.752559 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 365/758] 10.222.0.46 () {56 vars in 1111 bytes} [Mon Oct 10 11:47:22 2022] GET /identity/v3/services/network => generated 88 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:47:22.753416 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46680de7-93b9-46da-a317-17d85510e15e None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:22.759023 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 394/759] 10.222.0.46 () {56 vars in 1109 bytes} [Mon Oct 10 11:47:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:47:22.760068 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911ac37a-20d0-49e0-8390-6c022551e706 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:22.848808 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:22.915456 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:22.923784 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46680de7-93b9-46da-a317-17d85510e15e None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:22.956661 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911ac37a-20d0-49e0-8390-6c022551e706 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:22.979291 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d866ea-2c57-42dd-9941-55aa620fd96c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:22.980012 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d866ea-2c57-42dd-9941-55aa620fd96c None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:22.980302 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d866ea-2c57-42dd-9941-55aa620fd96c None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:22.980580 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d866ea-2c57-42dd-9941-55aa620fd96c None admin] PATH_INFO: `/v3/domains` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:22.981297 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d866ea-2c57-42dd-9941-55aa620fd96c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:47:22.981607 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d866ea-2c57-42dd-9941-55aa620fd96c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:22.983040 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d866ea-2c57-42dd-9941-55aa620fd96c None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:22.997732 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 366/760] 10.222.0.46 () {56 vars in 1109 bytes} [Mon Oct 10 11:47:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:47:22.998608 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d866ea-2c57-42dd-9941-55aa620fd96c None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:23.031505 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6d8eaf-5535-4c1a-baa4-9cb873ea40bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:23.032236 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6d8eaf-5535-4c1a-baa4-9cb873ea40bc None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:23.032582 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6d8eaf-5535-4c1a-baa4-9cb873ea40bc None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:23.032996 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6d8eaf-5535-4c1a-baa4-9cb873ea40bc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:23.131893 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6d8eaf-5535-4c1a-baa4-9cb873ea40bc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:23.133723 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6d8eaf-5535-4c1a-baa4-9cb873ea40bc None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:23.169952 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d866ea-2c57-42dd-9941-55aa620fd96c None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:23.185467 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:23.218164 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512fbd8f-9ca3-41a0-ab0e-d742c3397beb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:23.219355 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512fbd8f-9ca3-41a0-ab0e-d742c3397beb None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:23.219885 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512fbd8f-9ca3-41a0-ab0e-d742c3397beb None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:23.220938 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512fbd8f-9ca3-41a0-ab0e-d742c3397beb None admin] PATH_INFO: `/v3/services` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:23.223236 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512fbd8f-9ca3-41a0-ab0e-d742c3397beb None admin] RBAC: Adding query filter params (name=network) {{(pid=81676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:47:23.223236 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512fbd8f-9ca3-41a0-ab0e-d742c3397beb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:23.226504 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512fbd8f-9ca3-41a0-ab0e-d742c3397beb None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:23.237449 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 367/761] 10.222.0.46 () {56 vars in 1112 bytes} [Mon Oct 10 11:47:22 2022] GET /identity/v3/services?name=network => generated 124 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:47:23.239136 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512fbd8f-9ca3-41a0-ab0e-d742c3397beb None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:23.267662 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:23.299018 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 395/762] 10.222.0.46 () {58 vars in 1282 bytes} [Mon Oct 10 11:47:22 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 533 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 10 11:47:23.299018 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6d8eaf-5535-4c1a-baa4-9cb873ea40bc None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:23.401030 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512fbd8f-9ca3-41a0-ab0e-d742c3397beb None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:23.436105 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6d8eaf-5535-4c1a-baa4-9cb873ea40bc None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:23.438298 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd464674-dffd-45ec-af51-c62480e5bd13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:23.439110 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd464674-dffd-45ec-af51-c62480e5bd13 None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:23.439340 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd464674-dffd-45ec-af51-c62480e5bd13 None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:23.439613 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd464674-dffd-45ec-af51-c62480e5bd13 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:23.447152 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd464674-dffd-45ec-af51-c62480e5bd13 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:23.449742 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd464674-dffd-45ec-af51-c62480e5bd13 None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:23.465338 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application [None req-dd464674-dffd-45ec-af51-c62480e5bd13 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 10 11:47:23.465338 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:47:23.465338 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:47:23.465338 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:47:23.465338 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:47:23.465338 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:47:23.465338 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:47:23.465338 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:47:23.465338 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:47:23.465338 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:47:23.465338 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:47:23.465338 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:47:23.465338 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 10 11:47:23.465338 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 10 11:47:23.465338 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 10 11:47:23.476150 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 10 11:47:23.476150 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:47:23.476150 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:47:23.476150 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:47:23.476150 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:47:23.476150 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:47:23.476150 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:47:23.476150 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:47:23.476150 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application with Lock( Oct 10 11:47:23.476150 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:47:23.476150 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:47:23.476150 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:47:23.476150 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:47:23.476150 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:47:23.476150 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:47:23.476150 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:47:23.476150 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:47:23.479007 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 10 11:47:23.479007 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 10 11:47:23.479007 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 10 11:47:23.479007 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 10 11:47:23.479007 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 10 11:47:23.479007 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 10 11:47:23.479007 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 10 11:47:23.479007 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Oct 10 11:47:23.479007 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 368/763] 10.222.0.46 () {56 vars in 1111 bytes} [Mon Oct 10 11:47:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:47:23.479007 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd464674-dffd-45ec-af51-c62480e5bd13 None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:23.492967 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c0b619-d2df-4012-bf71-d8ba53006cec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:23.494101 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c0b619-d2df-4012-bf71-d8ba53006cec None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:23.494592 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c0b619-d2df-4012-bf71-d8ba53006cec None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:23.495359 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c0b619-d2df-4012-bf71-d8ba53006cec None admin] PATH_INFO: `/v3/users/nova` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:23.509262 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c0b619-d2df-4012-bf71-d8ba53006cec None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:23.511949 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c0b619-d2df-4012-bf71-d8ba53006cec None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:23.516069 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:23.535454 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application [None req-49c0b619-d2df-4012-bf71-d8ba53006cec None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 10 11:47:23.535454 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:47:23.535454 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:47:23.535454 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:47:23.535454 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:47:23.535454 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:47:23.535454 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:47:23.535454 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:47:23.535454 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:47:23.535454 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:47:23.535454 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:47:23.535454 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:47:23.535454 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 10 11:47:23.535454 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 10 11:47:23.535454 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 10 11:47:23.535454 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 10 11:47:23.536492 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:47:23.536492 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:47:23.536492 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 10 11:47:23.536492 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 10 11:47:23.536492 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 10 11:47:23.536492 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 10 11:47:23.536492 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:47:23.536492 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:47:23.536492 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:47:23.536492 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:47:23.536492 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:47:23.536492 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application with Lock( Oct 10 11:47:23.536492 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:47:23.536492 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:47:23.536492 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:47:23.536492 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:47:23.536492 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:47:23.536492 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:47:23.537411 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:47:23.537411 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:47:23.537411 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 10 11:47:23.537411 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 10 11:47:23.537411 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 10 11:47:23.537411 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 10 11:47:23.537411 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 10 11:47:23.537411 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 10 11:47:23.537411 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 10 11:47:23.537411 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Oct 10 11:47:23.537940 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 396/764] 10.222.0.46 () {56 vars in 1093 bytes} [Mon Oct 10 11:47:23 2022] GET /identity/v3/users/nova => generated 82 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:47:23.539341 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c0b619-d2df-4012-bf71-d8ba53006cec None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:23.589081 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd464674-dffd-45ec-af51-c62480e5bd13 None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:23.614905 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:23.627381 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d749b96a-a7ab-4ea8-b579-472f6a9b4a82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:23.632366 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d749b96a-a7ab-4ea8-b579-472f6a9b4a82 None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:23.632366 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d749b96a-a7ab-4ea8-b579-472f6a9b4a82 None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:23.632366 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d749b96a-a7ab-4ea8-b579-472f6a9b4a82 None admin] PATH_INFO: `/v3/services` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:23.632366 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d749b96a-a7ab-4ea8-b579-472f6a9b4a82 None admin] RBAC: Adding query filter params (type=network) {{(pid=81676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:47:23.632366 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d749b96a-a7ab-4ea8-b579-472f6a9b4a82 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:23.632366 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d749b96a-a7ab-4ea8-b579-472f6a9b4a82 None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:23.636384 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 369/765] 10.222.0.46 () {56 vars in 1112 bytes} [Mon Oct 10 11:47:23 2022] GET /identity/v3/services?type=network => generated 350 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:47:23.637773 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4a1a35-21eb-47ee-b0de-06459d632563 None None] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:23.638011 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4a1a35-21eb-47ee-b0de-06459d632563 None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:23.638290 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4a1a35-21eb-47ee-b0de-06459d632563 None None] PATH_INFO: `/` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:23.639876 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 370/766] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 10 11:47:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:47:23.643793 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4a1a35-21eb-47ee-b0de-06459d632563 None None] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:23.710293 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c0b619-d2df-4012-bf71-d8ba53006cec None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:23.760883 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2d86a6-2b30-4a8e-add7-8674eb7b6dd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:23.761566 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2d86a6-2b30-4a8e-add7-8674eb7b6dd0 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:23.761925 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2d86a6-2b30-4a8e-add7-8674eb7b6dd0 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:23.762322 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2d86a6-2b30-4a8e-add7-8674eb7b6dd0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:23.774856 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2d86a6-2b30-4a8e-add7-8674eb7b6dd0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:23.776509 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2d86a6-2b30-4a8e-add7-8674eb7b6dd0 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:23.790547 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:23.798001 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4a1a35-21eb-47ee-b0de-06459d632563 None None] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:23.804469 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application [None req-6a2d86a6-2b30-4a8e-add7-8674eb7b6dd0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:47:23.804469 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:47:23.804469 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:47:23.804469 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:47:23.804469 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:47:23.804469 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:47:23.804469 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:47:23.804469 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:47:23.804469 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:47:23.804469 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:47:23.804469 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:47:23.804469 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:47:23.804469 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 10 11:47:23.804469 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 10 11:47:23.804469 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 10 11:47:23.804469 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 10 11:47:23.806194 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:47:23.806194 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:47:23.806194 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:47:23.806194 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:47:23.806194 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:47:23.806194 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:47:23.806194 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:47:23.806194 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application with Lock( Oct 10 11:47:23.806194 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:47:23.806194 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:47:23.806194 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:47:23.806194 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:47:23.806194 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:47:23.806194 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:47:23.806194 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:47:23.806194 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:47:23.806194 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 10 11:47:23.807969 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 10 11:47:23.807969 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:47:23.807969 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Oct 10 11:47:23.807969 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 397/767] 10.222.0.46 () {56 vars in 1108 bytes} [Mon Oct 10 11:47:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:47:23.807969 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2d86a6-2b30-4a8e-add7-8674eb7b6dd0 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:23.850613 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d10eff-e86b-46ee-89dd-d521df41fecf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:23.851605 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d10eff-e86b-46ee-89dd-d521df41fecf None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:23.852028 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d10eff-e86b-46ee-89dd-d521df41fecf None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:23.852462 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d10eff-e86b-46ee-89dd-d521df41fecf None admin] PATH_INFO: `/v3/projects` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:23.853668 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d10eff-e86b-46ee-89dd-d521df41fecf None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:47:23.854083 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d10eff-e86b-46ee-89dd-d521df41fecf None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:23.857105 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d10eff-e86b-46ee-89dd-d521df41fecf None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:23.877418 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 371/768] 10.222.0.46 () {56 vars in 1148 bytes} [Mon Oct 10 11:47:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:47:23.878652 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d10eff-e86b-46ee-89dd-d521df41fecf None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:23.945425 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:23.956813 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2d86a6-2b30-4a8e-add7-8674eb7b6dd0 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:23.998494 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e6ab72-1f16-4e8e-8a2b-d78bc2efecfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:23.999255 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e6ab72-1f16-4e8e-8a2b-d78bc2efecfa None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:23.999605 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e6ab72-1f16-4e8e-8a2b-d78bc2efecfa None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:23.999941 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e6ab72-1f16-4e8e-8a2b-d78bc2efecfa None admin] PATH_INFO: `/v3/users` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:24.000764 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e6ab72-1f16-4e8e-8a2b-d78bc2efecfa None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:47:24.001155 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e6ab72-1f16-4e8e-8a2b-d78bc2efecfa None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:24.002961 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e6ab72-1f16-4e8e-8a2b-d78bc2efecfa None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:24.005582 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d10eff-e86b-46ee-89dd-d521df41fecf None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:24.026849 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 398/769] 10.222.0.46 () {56 vars in 1133 bytes} [Mon Oct 10 11:47:23 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:47:24.029447 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98be567-ede5-4a18-9dd1-39446fab78e7 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:24.029805 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98be567-ede5-4a18-9dd1-39446fab78e7 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:24.030178 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98be567-ede5-4a18-9dd1-39446fab78e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:24.051658 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-b98be567-ede5-4a18-9dd1-39446fab78e7 None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:47:24.051931 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ab5ae1-2163-4f9a-bdd1-942a00bdb6fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:24.052623 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ab5ae1-2163-4f9a-bdd1-942a00bdb6fd None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:24.053007 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ab5ae1-2163-4f9a-bdd1-942a00bdb6fd None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:24.053302 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-b98be567-ede5-4a18-9dd1-39446fab78e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:47:24.053475 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ab5ae1-2163-4f9a-bdd1-942a00bdb6fd None admin] PATH_INFO: `/v3/endpoints` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:24.054303 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ab5ae1-2163-4f9a-bdd1-942a00bdb6fd None admin] RBAC: Adding query filter params (interface=public, service_id=2a9a78a563cf494d8530485b2413acc3, region_id=RegionOne) {{(pid=81676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:47:24.054726 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ab5ae1-2163-4f9a-bdd1-942a00bdb6fd None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:24.056335 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ab5ae1-2163-4f9a-bdd1-942a00bdb6fd None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:24.063090 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 372/770] 10.222.0.46 () {56 vars in 1251 bytes} [Mon Oct 10 11:47:23 2022] GET /identity/v3/endpoints?service_id=2a9a78a563cf494d8530485b2413acc3&interface=public®ion_id=RegionOne => generated 194 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:47:24.064176 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ab5ae1-2163-4f9a-bdd1-942a00bdb6fd None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:24.096912 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:24.140085 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 399/771] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 10 11:47:24 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 10 11:47:24.142362 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98be567-ede5-4a18-9dd1-39446fab78e7 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:24.210904 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ab5ae1-2163-4f9a-bdd1-942a00bdb6fd None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:24.257918 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e45bd75-a09f-49ff-899f-a6ccb01689c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:24.259278 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e45bd75-a09f-49ff-899f-a6ccb01689c3 None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:24.259734 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e45bd75-a09f-49ff-899f-a6ccb01689c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:24.260306 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e45bd75-a09f-49ff-899f-a6ccb01689c3 None admin] PATH_INFO: `/v3/domains` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:24.261153 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e45bd75-a09f-49ff-899f-a6ccb01689c3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:47:24.261572 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e45bd75-a09f-49ff-899f-a6ccb01689c3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:24.263270 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e45bd75-a09f-49ff-899f-a6ccb01689c3 None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:24.286758 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 373/772] 10.222.0.46 () {56 vars in 1109 bytes} [Mon Oct 10 11:47:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:47:24.286758 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e45bd75-a09f-49ff-899f-a6ccb01689c3 None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:24.298223 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:24.337288 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98be567-ede5-4a18-9dd1-39446fab78e7 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:24.390957 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a7441d-e641-4258-aa5d-da9500b79b06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:24.391789 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a7441d-e641-4258-aa5d-da9500b79b06 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:24.392321 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a7441d-e641-4258-aa5d-da9500b79b06 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:24.392795 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a7441d-e641-4258-aa5d-da9500b79b06 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:24.394086 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a7441d-e641-4258-aa5d-da9500b79b06 None admin] RBAC: Adding query filter params (role.id=0d822a0674b6495fac623877a3b38025, scope.project.id=4ad3772f7b6a47c28b7ba37a9316a3c1, user.id=15ef3d53ea484237adb0b3d64fdc5480) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:47:24.394605 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a7441d-e641-4258-aa5d-da9500b79b06 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:24.397120 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a7441d-e641-4258-aa5d-da9500b79b06 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:24.407677 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 400/773] 10.222.0.46 () {56 vars in 1374 bytes} [Mon Oct 10 11:47:24 2022] GET /identity/v3/role_assignments?user.id=15ef3d53ea484237adb0b3d64fdc5480&scope.project.id=4ad3772f7b6a47c28b7ba37a9316a3c1&role.id=0d822a0674b6495fac623877a3b38025 => generated 259 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:47:24.409051 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a7441d-e641-4258-aa5d-da9500b79b06 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:24.433151 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:24.472705 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e45bd75-a09f-49ff-899f-a6ccb01689c3 None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:24.565587 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881b20cb-f644-4a23-8c94-ea500ad17388 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:24.566763 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881b20cb-f644-4a23-8c94-ea500ad17388 None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:24.567325 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881b20cb-f644-4a23-8c94-ea500ad17388 None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:24.567888 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881b20cb-f644-4a23-8c94-ea500ad17388 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:24.583914 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881b20cb-f644-4a23-8c94-ea500ad17388 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:24.585741 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881b20cb-f644-4a23-8c94-ea500ad17388 None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:24.603313 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application [None req-881b20cb-f644-4a23-8c94-ea500ad17388 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:47:24.603313 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:47:24.603313 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:47:24.603313 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:47:24.603313 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:47:24.603313 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:47:24.603313 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:47:24.603313 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:47:24.603313 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:47:24.603313 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:47:24.603313 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:47:24.603313 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:47:24.603313 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 10 11:47:24.603313 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 10 11:47:24.603313 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 10 11:47:24.603313 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 10 11:47:24.604469 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:47:24.604469 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:47:24.604469 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:47:24.604469 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:47:24.604469 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:47:24.604469 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:47:24.604469 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:47:24.604469 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application with Lock( Oct 10 11:47:24.604469 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:47:24.604469 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:47:24.604469 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:47:24.604469 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:47:24.604469 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:47:24.604469 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:47:24.604469 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:47:24.604469 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:47:24.604469 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 10 11:47:24.605429 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 10 11:47:24.605429 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:47:24.605429 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Oct 10 11:47:24.605429 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 374/774] 10.222.0.46 () {56 vars in 1108 bytes} [Mon Oct 10 11:47:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 325 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:47:24.605653 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881b20cb-f644-4a23-8c94-ea500ad17388 None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:24.623887 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a7441d-e641-4258-aa5d-da9500b79b06 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:24.665758 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9658f63a-ef94-45d1-abc3-26612baf8770 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:24.666651 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9658f63a-ef94-45d1-abc3-26612baf8770 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:24.667162 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9658f63a-ef94-45d1-abc3-26612baf8770 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:24.667419 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9658f63a-ef94-45d1-abc3-26612baf8770 None admin] PATH_INFO: `/v3/services` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:24.668051 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9658f63a-ef94-45d1-abc3-26612baf8770 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:24.670042 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9658f63a-ef94-45d1-abc3-26612baf8770 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:24.677224 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 401/775] 10.222.0.46 () {56 vars in 1087 bytes} [Mon Oct 10 11:47:24 2022] GET /identity/v3/services => generated 772 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:47:24.678786 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9658f63a-ef94-45d1-abc3-26612baf8770 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:24.699222 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:24.795838 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881b20cb-f644-4a23-8c94-ea500ad17388 None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:24.869264 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38fc0736-0ba6-4e47-8307-3b686c5aa5bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:24.870706 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fc0736-0ba6-4e47-8307-3b686c5aa5bc None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:24.871232 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fc0736-0ba6-4e47-8307-3b686c5aa5bc None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:24.872245 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fc0736-0ba6-4e47-8307-3b686c5aa5bc None admin] PATH_INFO: `/v3/projects/service` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:24.886565 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38fc0736-0ba6-4e47-8307-3b686c5aa5bc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:24.889637 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38fc0736-0ba6-4e47-8307-3b686c5aa5bc None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:24.890885 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:24.907281 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9658f63a-ef94-45d1-abc3-26612baf8770 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:24.915572 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application [None req-38fc0736-0ba6-4e47-8307-3b686c5aa5bc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 10 11:47:24.915572 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:47:24.915572 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:47:24.915572 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:47:24.915572 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:47:24.915572 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:47:24.915572 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:47:24.915572 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:47:24.915572 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:47:24.915572 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:47:24.915572 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:47:24.915572 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:47:24.915572 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 10 11:47:24.915572 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 10 11:47:24.915572 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 10 11:47:24.917428 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 10 11:47:24.917428 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:47:24.917428 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:47:24.917428 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:47:24.917428 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:47:24.917428 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:47:24.917428 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:47:24.917428 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:47:24.917428 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application with Lock( Oct 10 11:47:24.917428 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:47:24.917428 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:47:24.917428 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:47:24.917428 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:47:24.917428 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:47:24.917428 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:47:24.917428 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:47:24.917428 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:47:24.920434 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 10 11:47:24.920434 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 10 11:47:24.920434 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 10 11:47:24.920434 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 10 11:47:24.920434 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 10 11:47:24.920434 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 10 11:47:24.920434 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 10 11:47:24.920434 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Oct 10 11:47:24.930729 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 375/776] 10.222.0.46 () {56 vars in 1111 bytes} [Mon Oct 10 11:47:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 325 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:47:24.932485 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38fc0736-0ba6-4e47-8307-3b686c5aa5bc None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:24.975524 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783370ed-879f-4194-873e-bf1864520c30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:24.976185 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783370ed-879f-4194-873e-bf1864520c30 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:24.976473 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783370ed-879f-4194-873e-bf1864520c30 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:24.976723 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783370ed-879f-4194-873e-bf1864520c30 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:24.977323 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783370ed-879f-4194-873e-bf1864520c30 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:24.978820 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783370ed-879f-4194-873e-bf1864520c30 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:24.990323 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application [None req-783370ed-879f-4194-873e-bf1864520c30 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 10 11:47:24.990323 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:47:24.990323 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 10 11:47:24.990323 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 10 11:47:24.990323 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 10 11:47:24.990323 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 10 11:47:24.990323 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 10 11:47:24.990323 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 10 11:47:24.990323 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 10 11:47:24.990323 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Oct 10 11:47:24.990323 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 10 11:47:24.990323 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Oct 10 11:47:24.990323 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:47:24.990323 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:47:24.990323 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:47:24.990323 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:47:24.990323 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:47:24.992428 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:47:24.992428 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:47:24.992428 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:47:24.992428 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:47:24.992428 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:47:24.992428 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:47:24.992428 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 10 11:47:24.992428 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 10 11:47:24.992428 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 10 11:47:24.992428 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 10 11:47:24.992428 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:47:24.992428 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:47:24.992428 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:47:24.992428 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:47:24.992428 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:47:24.992428 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:47:24.994114 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:47:24.994114 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application with Lock( Oct 10 11:47:24.994114 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:47:24.994114 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:47:24.994114 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:47:24.994114 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:47:24.994114 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:47:24.994114 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:47:24.994114 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:47:24.994114 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:47:24.994114 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 10 11:47:24.994114 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 10 11:47:24.994114 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 10 11:47:24.994114 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Oct 10 11:47:24.994114 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 402/777] 10.222.0.46 () {56 vars in 1129 bytes} [Mon Oct 10 11:47:24 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 314 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:47:24.999099 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783370ed-879f-4194-873e-bf1864520c30 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:25.139011 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38fc0736-0ba6-4e47-8307-3b686c5aa5bc None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:25.145452 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:25.183009 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783370ed-879f-4194-873e-bf1864520c30 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:25.206789 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058cdfde-2cf8-4f85-8286-08562dd398f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:25.207587 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058cdfde-2cf8-4f85-8286-08562dd398f0 None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:25.207967 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058cdfde-2cf8-4f85-8286-08562dd398f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:25.208338 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058cdfde-2cf8-4f85-8286-08562dd398f0 None admin] PATH_INFO: `/v3/domains` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:25.209120 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058cdfde-2cf8-4f85-8286-08562dd398f0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:47:25.209962 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058cdfde-2cf8-4f85-8286-08562dd398f0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:25.211232 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058cdfde-2cf8-4f85-8286-08562dd398f0 None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:25.234733 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 376/778] 10.222.0.46 () {56 vars in 1109 bytes} [Mon Oct 10 11:47:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:47:25.234733 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058cdfde-2cf8-4f85-8286-08562dd398f0 None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:25.279459 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f835107c-ba23-4c3e-86af-6437de2a3062 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:25.280820 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f835107c-ba23-4c3e-86af-6437de2a3062 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:25.281632 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f835107c-ba23-4c3e-86af-6437de2a3062 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:25.283429 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f835107c-ba23-4c3e-86af-6437de2a3062 None admin] PATH_INFO: `/v3/projects` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:25.285123 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f835107c-ba23-4c3e-86af-6437de2a3062 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:47:25.286576 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f835107c-ba23-4c3e-86af-6437de2a3062 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:25.290999 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f835107c-ba23-4c3e-86af-6437de2a3062 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:25.305040 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:25.331763 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 403/779] 10.222.0.46 () {56 vars in 1148 bytes} [Mon Oct 10 11:47:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 334 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:47:25.332158 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f835107c-ba23-4c3e-86af-6437de2a3062 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:25.368475 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058cdfde-2cf8-4f85-8286-08562dd398f0 None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:25.413450 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2faf79-081c-4fc7-8c8b-0e48a0bd574b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:25.413970 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2faf79-081c-4fc7-8c8b-0e48a0bd574b None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:25.414247 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2faf79-081c-4fc7-8c8b-0e48a0bd574b None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:25.414522 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2faf79-081c-4fc7-8c8b-0e48a0bd574b None admin] PATH_INFO: `/v3/services` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:25.415245 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2faf79-081c-4fc7-8c8b-0e48a0bd574b None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:47:25.415541 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2faf79-081c-4fc7-8c8b-0e48a0bd574b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:25.416926 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2faf79-081c-4fc7-8c8b-0e48a0bd574b None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:25.423402 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 377/780] 10.222.0.46 () {56 vars in 1124 bytes} [Mon Oct 10 11:47:25 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:47:25.424029 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2faf79-081c-4fc7-8c8b-0e48a0bd574b None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:25.547359 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f835107c-ba23-4c3e-86af-6437de2a3062 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:25.617139 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2faf79-081c-4fc7-8c8b-0e48a0bd574b None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:25.640344 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:25.645997 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:25.655405 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8568315a-fc4c-4174-a360-4921ddd53bb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:25.656065 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8568315a-fc4c-4174-a360-4921ddd53bb0 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:25.656339 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8568315a-fc4c-4174-a360-4921ddd53bb0 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:25.656609 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8568315a-fc4c-4174-a360-4921ddd53bb0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:25.669801 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543f9619-0b8d-4fcc-8787-811aa9902996 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:25.670335 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543f9619-0b8d-4fcc-8787-811aa9902996 None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:25.670622 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543f9619-0b8d-4fcc-8787-811aa9902996 None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:25.670974 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543f9619-0b8d-4fcc-8787-811aa9902996 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:25.671737 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543f9619-0b8d-4fcc-8787-811aa9902996 None admin] RBAC: Adding query filter params (role.id=3c35be38787740f4834c728a2ce73b6b, scope.project.id=4ad3772f7b6a47c28b7ba37a9316a3c1, user.id=d6430593eecf45d8b891c3b55de4a944) {{(pid=81676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:47:25.672045 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543f9619-0b8d-4fcc-8787-811aa9902996 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:25.673649 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543f9619-0b8d-4fcc-8787-811aa9902996 None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:25.674879 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8568315a-fc4c-4174-a360-4921ddd53bb0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:25.679848 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8568315a-fc4c-4174-a360-4921ddd53bb0 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:25.685834 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 378/781] 10.222.0.46 () {56 vars in 1374 bytes} [Mon Oct 10 11:47:25 2022] GET /identity/v3/role_assignments?user.id=d6430593eecf45d8b891c3b55de4a944&scope.project.id=4ad3772f7b6a47c28b7ba37a9316a3c1&role.id=3c35be38787740f4834c728a2ce73b6b => generated 259 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:47:25.685834 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543f9619-0b8d-4fcc-8787-811aa9902996 None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:25.720585 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application [None req-8568315a-fc4c-4174-a360-4921ddd53bb0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 10 11:47:25.720585 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:47:25.720585 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:47:25.720585 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:47:25.720585 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:47:25.720585 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:47:25.720585 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:47:25.720585 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:47:25.720585 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:47:25.720585 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:47:25.720585 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:47:25.720585 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:47:25.720585 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 10 11:47:25.720585 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 10 11:47:25.720585 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 10 11:47:25.722133 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 10 11:47:25.722133 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:47:25.722133 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:47:25.722133 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:47:25.722133 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:47:25.722133 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:47:25.722133 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:47:25.722133 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:47:25.722133 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application with Lock( Oct 10 11:47:25.722133 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:47:25.722133 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:47:25.722133 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:47:25.722133 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:47:25.722133 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:47:25.722133 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:47:25.722133 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:47:25.722133 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:47:25.724227 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 10 11:47:25.724227 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 10 11:47:25.724227 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 10 11:47:25.724227 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 10 11:47:25.724227 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 10 11:47:25.724227 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 10 11:47:25.724227 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 10 11:47:25.724227 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Oct 10 11:47:25.728156 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 404/782] 10.222.0.46 () {56 vars in 1111 bytes} [Mon Oct 10 11:47:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 396 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:47:25.738797 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8568315a-fc4c-4174-a360-4921ddd53bb0 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:25.995808 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8568315a-fc4c-4174-a360-4921ddd53bb0 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:26.025159 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543f9619-0b8d-4fcc-8787-811aa9902996 None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:26.066993 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2423a2d9-2eb7-4836-94fd-5819932054ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:26.068488 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2423a2d9-2eb7-4836-94fd-5819932054ef None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:26.069256 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2423a2d9-2eb7-4836-94fd-5819932054ef None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:26.069999 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2423a2d9-2eb7-4836-94fd-5819932054ef None admin] PATH_INFO: `/v3/projects` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:26.071879 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2423a2d9-2eb7-4836-94fd-5819932054ef None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:47:26.073177 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2423a2d9-2eb7-4836-94fd-5819932054ef None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:26.076976 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2423a2d9-2eb7-4836-94fd-5819932054ef None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:26.102080 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9167bbb-66f9-4734-8fcb-25334362e5d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:26.102706 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9167bbb-66f9-4734-8fcb-25334362e5d0 None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:26.103174 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9167bbb-66f9-4734-8fcb-25334362e5d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:26.103545 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9167bbb-66f9-4734-8fcb-25334362e5d0 None admin] PATH_INFO: `/v3/services` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:26.104303 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9167bbb-66f9-4734-8fcb-25334362e5d0 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:47:26.105314 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 405/783] 10.222.0.46 () {56 vars in 1148 bytes} [Mon Oct 10 11:47:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 367 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:47:26.108991 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9167bbb-66f9-4734-8fcb-25334362e5d0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:26.108991 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9167bbb-66f9-4734-8fcb-25334362e5d0 None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:26.116009 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2423a2d9-2eb7-4836-94fd-5819932054ef None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:26.124773 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 379/784] 10.222.0.46 () {56 vars in 1124 bytes} [Mon Oct 10 11:47:25 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 442 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:47:26.132517 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9167bbb-66f9-4734-8fcb-25334362e5d0 None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:26.237475 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:26.259611 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:26.345269 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2423a2d9-2eb7-4836-94fd-5819932054ef None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:26.413767 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2274433a-0e9c-424f-976d-a01272c394c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:26.414928 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2274433a-0e9c-424f-976d-a01272c394c1 None admin] REQUEST_METHOD: `PUT` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:26.415406 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2274433a-0e9c-424f-976d-a01272c394c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:26.415880 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2274433a-0e9c-424f-976d-a01272c394c1 None admin] PATH_INFO: `/v3/projects/4ad3772f7b6a47c28b7ba37a9316a3c1/users/5f130bbb69e546ce99b4e14de81d7e54/roles/0d822a0674b6495fac623877a3b38025` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:26.419886 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2274433a-0e9c-424f-976d-a01272c394c1 None admin] RBAC: Authorizing `identity:create_grant(project_id=4ad3772f7b6a47c28b7ba37a9316a3c1, user_id=5f130bbb69e546ce99b4e14de81d7e54, role_id=0d822a0674b6495fac623877a3b38025)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:26.423259 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2274433a-0e9c-424f-976d-a01272c394c1 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:26.444101 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9167bbb-66f9-4734-8fcb-25334362e5d0 None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:26.516156 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da8858c-dd4a-4e1f-966e-1dad04a63a64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:26.517285 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da8858c-dd4a-4e1f-966e-1dad04a63a64 None admin] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:26.517806 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da8858c-dd4a-4e1f-966e-1dad04a63a64 None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:26.518273 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da8858c-dd4a-4e1f-966e-1dad04a63a64 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:26.519726 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da8858c-dd4a-4e1f-966e-1dad04a63a64 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:26.522642 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da8858c-dd4a-4e1f-966e-1dad04a63a64 None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:26.582189 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 406/785] 10.222.0.46 () {58 vars in 1439 bytes} [Mon Oct 10 11:47:26 2022] PUT /identity/v3/projects/4ad3772f7b6a47c28b7ba37a9316a3c1/users/5f130bbb69e546ce99b4e14de81d7e54/roles/0d822a0674b6495fac623877a3b38025 => generated 0 bytes in 467 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 10 11:47:26.639926 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 380/786] 10.222.0.46 () {60 vars in 1144 bytes} [Mon Oct 10 11:47:26 2022] POST /identity/v3/endpoints => generated 346 bytes in 508 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 10 11:47:27.344971 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57021344-1385-46c5-9963-531072fb5056 None None] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:27.345763 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57021344-1385-46c5-9963-531072fb5056 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:27.345901 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57021344-1385-46c5-9963-531072fb5056 None None] PATH_INFO: `/` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:27.348631 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 407/787] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 10 11:47:27 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:47:27.355148 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62155787-f4c6-450d-9291-dbf68201af04 None None] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:27.355614 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62155787-f4c6-450d-9291-dbf68201af04 None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:27.356132 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62155787-f4c6-450d-9291-dbf68201af04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:27.388422 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.auth.core [None req-62155787-f4c6-450d-9291-dbf68201af04 None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:47:27.390382 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-62155787-f4c6-450d-9291-dbf68201af04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:47:27.532657 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 381/788] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 10 11:47:27 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 10 11:47:28.131594 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57021344-1385-46c5-9963-531072fb5056 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:28.278395 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243dca32-34fe-4744-b4bb-eb73e0a2dae1 None None] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:28.279304 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243dca32-34fe-4744-b4bb-eb73e0a2dae1 None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:28.279728 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243dca32-34fe-4744-b4bb-eb73e0a2dae1 None None] PATH_INFO: `/` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:28.283599 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 382/789] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 10 11:47:28 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 10 11:47:28.309676 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa724c2-a433-4345-920e-2840dd36083c None None] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:28.309970 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa724c2-a433-4345-920e-2840dd36083c None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:28.310215 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa724c2-a433-4345-920e-2840dd36083c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:28.348832 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.auth.core [None req-8aa724c2-a433-4345-920e-2840dd36083c None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:47:28.351376 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-8aa724c2-a433-4345-920e-2840dd36083c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:47:28.356976 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57021344-1385-46c5-9963-531072fb5056 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:28.392114 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:28.403707 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:28.468447 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a247e182-b5df-45ac-b2dc-4e37f8e5d993 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:28.469106 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a247e182-b5df-45ac-b2dc-4e37f8e5d993 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:28.469388 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a247e182-b5df-45ac-b2dc-4e37f8e5d993 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:28.469657 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a247e182-b5df-45ac-b2dc-4e37f8e5d993 None admin] PATH_INFO: `/v3/services/network` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:28.470251 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a247e182-b5df-45ac-b2dc-4e37f8e5d993 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:28.471781 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a247e182-b5df-45ac-b2dc-4e37f8e5d993 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:28.487537 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application [None req-a247e182-b5df-45ac-b2dc-4e37f8e5d993 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Oct 10 11:47:28.487537 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:47:28.487537 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 10 11:47:28.487537 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 10 11:47:28.487537 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 10 11:47:28.487537 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 10 11:47:28.487537 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 10 11:47:28.487537 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 10 11:47:28.487537 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 10 11:47:28.487537 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Oct 10 11:47:28.487537 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 10 11:47:28.487537 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Oct 10 11:47:28.487537 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:47:28.487537 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:47:28.487537 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:47:28.487537 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:47:28.487537 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:47:28.489242 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:47:28.489242 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:47:28.489242 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:47:28.489242 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:47:28.489242 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:47:28.489242 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:47:28.489242 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 10 11:47:28.489242 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 10 11:47:28.489242 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 10 11:47:28.489242 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 10 11:47:28.489242 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:47:28.489242 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:47:28.489242 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:47:28.489242 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:47:28.489242 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:47:28.489242 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:47:28.491440 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:47:28.491440 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application with Lock( Oct 10 11:47:28.491440 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:47:28.491440 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:47:28.491440 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:47:28.491440 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:47:28.491440 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:47:28.491440 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:47:28.491440 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:47:28.491440 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:47:28.491440 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 10 11:47:28.491440 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 10 11:47:28.491440 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 10 11:47:28.491440 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Oct 10 11:47:28.492825 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 408/790] 10.222.0.46 () {56 vars in 1111 bytes} [Mon Oct 10 11:47:28 2022] GET /identity/v3/services/network => generated 88 bytes in 361 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:47:28.503376 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a247e182-b5df-45ac-b2dc-4e37f8e5d993 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:28.544460 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 383/791] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 10 11:47:28 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 10 11:47:28.688737 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a247e182-b5df-45ac-b2dc-4e37f8e5d993 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:28.760215 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddd3733-a2d2-4aff-a923-9910294c9170 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:28.761220 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddd3733-a2d2-4aff-a923-9910294c9170 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:28.761705 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddd3733-a2d2-4aff-a923-9910294c9170 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:28.762715 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddd3733-a2d2-4aff-a923-9910294c9170 None admin] PATH_INFO: `/v3/services` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:28.763840 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fddd3733-a2d2-4aff-a923-9910294c9170 None admin] RBAC: Adding query filter params (name=network) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:47:28.764373 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fddd3733-a2d2-4aff-a923-9910294c9170 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:28.767171 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fddd3733-a2d2-4aff-a923-9910294c9170 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:28.780641 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 409/792] 10.222.0.46 () {56 vars in 1112 bytes} [Mon Oct 10 11:47:28 2022] GET /identity/v3/services?name=network => generated 124 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:47:28.791853 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa724c2-a433-4345-920e-2840dd36083c None None] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:28.963217 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa724c2-a433-4345-920e-2840dd36083c None None] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:29.016803 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89f17b2-13ad-402c-89e8-64679b00b5a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:29.017472 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89f17b2-13ad-402c-89e8-64679b00b5a4 None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:29.017752 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89f17b2-13ad-402c-89e8-64679b00b5a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:29.018030 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89f17b2-13ad-402c-89e8-64679b00b5a4 None admin] PATH_INFO: `/v3/services` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:29.018702 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89f17b2-13ad-402c-89e8-64679b00b5a4 None admin] RBAC: Adding query filter params (type=network) {{(pid=81676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:47:29.019021 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89f17b2-13ad-402c-89e8-64679b00b5a4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:29.020457 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89f17b2-13ad-402c-89e8-64679b00b5a4 None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:29.029143 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 384/793] 10.222.0.46 () {56 vars in 1112 bytes} [Mon Oct 10 11:47:28 2022] GET /identity/v3/services?type=network => generated 350 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:47:29.037054 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddd3733-a2d2-4aff-a923-9910294c9170 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:29.102094 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89f17b2-13ad-402c-89e8-64679b00b5a4 None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:29.131556 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:29.162245 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:29.251056 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddd3733-a2d2-4aff-a923-9910294c9170 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:29.331157 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9619ddfd-23f5-4bf0-97bf-338a01dc1256 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:29.332190 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9619ddfd-23f5-4bf0-97bf-338a01dc1256 None admin] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:29.332677 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9619ddfd-23f5-4bf0-97bf-338a01dc1256 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:29.333157 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9619ddfd-23f5-4bf0-97bf-338a01dc1256 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:29.334451 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9619ddfd-23f5-4bf0-97bf-338a01dc1256 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:29.337226 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9619ddfd-23f5-4bf0-97bf-338a01dc1256 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:29.369227 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89f17b2-13ad-402c-89e8-64679b00b5a4 None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:29.452974 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e4ed97-be95-4341-8fa2-9293c7466123 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:29.456687 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e4ed97-be95-4341-8fa2-9293c7466123 None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:29.456687 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e4ed97-be95-4341-8fa2-9293c7466123 None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:29.456687 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e4ed97-be95-4341-8fa2-9293c7466123 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:29.472884 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 410/794] 10.222.0.46 () {60 vars in 1144 bytes} [Mon Oct 10 11:47:29 2022] POST /identity/v3/endpoints => generated 337 bytes in 437 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 11:47:29.496024 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e4ed97-be95-4341-8fa2-9293c7466123 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 10 11:47:29.507116 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application [None req-b4e4ed97-be95-4341-8fa2-9293c7466123 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 10 11:47:29.507116 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:47:29.507116 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:47:29.507116 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:47:29.507116 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:47:29.507116 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:47:29.507116 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:47:29.507116 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:47:29.507116 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:47:29.507116 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:47:29.507116 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:47:29.507116 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:47:29.507116 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 10 11:47:29.507116 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 10 11:47:29.507116 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 10 11:47:29.507116 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 10 11:47:29.509122 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 10 11:47:29.509122 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 10 11:47:29.509122 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 10 11:47:29.509122 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 10 11:47:29.509122 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:47:29.509122 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:47:29.509122 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:47:29.509122 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:47:29.509122 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:47:29.509122 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:47:29.509122 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:47:29.509122 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application with Lock( Oct 10 11:47:29.509122 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:47:29.509122 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:47:29.509122 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:47:29.509122 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:47:29.510997 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:47:29.510997 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:47:29.510997 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:47:29.510997 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:47:29.510997 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 10 11:47:29.510997 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 10 11:47:29.510997 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 10 11:47:29.510997 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 10 11:47:29.510997 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 10 11:47:29.510997 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 10 11:47:29.510997 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 10 11:47:29.510997 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Oct 10 11:47:29.513357 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 385/795] 10.222.0.46 () {56 vars in 1096 bytes} [Mon Oct 10 11:47:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 412 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 10 11:47:29.517545 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9619ddfd-23f5-4bf0-97bf-338a01dc1256 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:29.703186 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:29.728319 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9619ddfd-23f5-4bf0-97bf-338a01dc1256 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:29.780876 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8b81e6-c64a-431f-a4f7-19cf5cb2e785 None None] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:29.781294 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8b81e6-c64a-431f-a4f7-19cf5cb2e785 None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:29.781658 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8b81e6-c64a-431f-a4f7-19cf5cb2e785 None None] PATH_INFO: `/` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:29.787216 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 386/796] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 10 11:47:29 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:47:29.797213 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2931a1e4-31a8-4927-85fe-11609345d073 None None] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:29.798218 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2931a1e4-31a8-4927-85fe-11609345d073 None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:29.802189 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd312e25-6daf-4766-ad53-07a2280e3709 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:29.802867 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd312e25-6daf-4766-ad53-07a2280e3709 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:29.803157 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd312e25-6daf-4766-ad53-07a2280e3709 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:29.803414 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd312e25-6daf-4766-ad53-07a2280e3709 None admin] PATH_INFO: `/v3/roles` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:29.803906 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2931a1e4-31a8-4927-85fe-11609345d073 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:29.804661 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd312e25-6daf-4766-ad53-07a2280e3709 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:47:29.805204 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd312e25-6daf-4766-ad53-07a2280e3709 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:29.807880 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd312e25-6daf-4766-ad53-07a2280e3709 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:29.825921 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 411/797] 10.222.0.46 () {56 vars in 1099 bytes} [Mon Oct 10 11:47:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:47:29.833284 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd312e25-6daf-4766-ad53-07a2280e3709 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:29.852531 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.auth.core [None req-2931a1e4-31a8-4927-85fe-11609345d073 None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:47:29.855117 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-2931a1e4-31a8-4927-85fe-11609345d073 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:47:30.005012 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 387/798] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 10 11:47:29 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 10 11:47:30.074842 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd312e25-6daf-4766-ad53-07a2280e3709 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:30.155911 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf5bbd5-2c31-45ce-9af5-37143ad6107e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:30.157388 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf5bbd5-2c31-45ce-9af5-37143ad6107e None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:30.157795 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf5bbd5-2c31-45ce-9af5-37143ad6107e None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:30.158189 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf5bbd5-2c31-45ce-9af5-37143ad6107e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:30.183699 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf5bbd5-2c31-45ce-9af5-37143ad6107e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:30.187106 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf5bbd5-2c31-45ce-9af5-37143ad6107e None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:30.222622 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application [None req-6bf5bbd5-2c31-45ce-9af5-37143ad6107e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:47:30.222622 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:47:30.222622 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:47:30.222622 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:47:30.222622 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:47:30.222622 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:47:30.222622 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:47:30.222622 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:47:30.222622 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:47:30.222622 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:47:30.222622 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:47:30.222622 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:47:30.222622 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 10 11:47:30.222622 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 10 11:47:30.222622 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 10 11:47:30.222622 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 10 11:47:30.224462 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:47:30.224462 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:47:30.224462 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:47:30.224462 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:47:30.224462 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:47:30.224462 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:47:30.224462 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:47:30.224462 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application with Lock( Oct 10 11:47:30.224462 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:47:30.224462 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:47:30.224462 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:47:30.224462 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:47:30.224462 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:47:30.224462 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:47:30.224462 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:47:30.224462 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:47:30.224462 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 10 11:47:30.227243 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 10 11:47:30.227243 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:47:30.227243 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Oct 10 11:47:30.227243 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 412/799] 10.222.0.46 () {56 vars in 1108 bytes} [Mon Oct 10 11:47:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 393 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:47:30.230740 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2931a1e4-31a8-4927-85fe-11609345d073 None None] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:30.271366 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:30.451029 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2931a1e4-31a8-4927-85fe-11609345d073 None None] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:30.456303 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cd2290-0071-49a0-b6e9-5fc1a56d5e1c None None] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:30.456752 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cd2290-0071-49a0-b6e9-5fc1a56d5e1c None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:30.457206 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cd2290-0071-49a0-b6e9-5fc1a56d5e1c None None] PATH_INFO: `/` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:30.468994 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 413/800] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 10 11:47:30 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:47:30.477755 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667b4dfc-e2bd-4d4f-9df6-d2057e284e26 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:30.478537 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667b4dfc-e2bd-4d4f-9df6-d2057e284e26 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:30.478862 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667b4dfc-e2bd-4d4f-9df6-d2057e284e26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:30.503163 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:30.513869 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-667b4dfc-e2bd-4d4f-9df6-d2057e284e26 None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:47:30.516140 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-667b4dfc-e2bd-4d4f-9df6-d2057e284e26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:47:30.534901 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a3a7e4-c253-4fa9-8172-125331743fd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:30.535915 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a3a7e4-c253-4fa9-8172-125331743fd7 None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:30.536831 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a3a7e4-c253-4fa9-8172-125331743fd7 None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:30.537307 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a3a7e4-c253-4fa9-8172-125331743fd7 None admin] PATH_INFO: `/v3/domains` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:30.538648 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a3a7e4-c253-4fa9-8172-125331743fd7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:47:30.539212 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a3a7e4-c253-4fa9-8172-125331743fd7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:30.541843 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a3a7e4-c253-4fa9-8172-125331743fd7 None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:30.563293 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 388/801] 10.222.0.46 () {56 vars in 1109 bytes} [Mon Oct 10 11:47:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 333 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 11:47:30.565550 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdf997b-7c13-405e-b1a8-fc19f77bbe92 None None] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:30.566046 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdf997b-7c13-405e-b1a8-fc19f77bbe92 None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:30.566547 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdf997b-7c13-405e-b1a8-fc19f77bbe92 None None] PATH_INFO: `/` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:30.569156 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 389/802] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 10 11:47:30 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:47:30.571022 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdf997b-7c13-405e-b1a8-fc19f77bbe92 None None] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:30.638726 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 414/803] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 10 11:47:30 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 10 11:47:30.638726 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667b4dfc-e2bd-4d4f-9df6-d2057e284e26 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:30.675214 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:30.714473 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdf997b-7c13-405e-b1a8-fc19f77bbe92 None None] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:30.771782 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a685285-3ba0-4b03-89a8-c016b2d90111 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:30.772773 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a685285-3ba0-4b03-89a8-c016b2d90111 None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:30.773214 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a685285-3ba0-4b03-89a8-c016b2d90111 None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:30.773651 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a685285-3ba0-4b03-89a8-c016b2d90111 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:30.790227 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a685285-3ba0-4b03-89a8-c016b2d90111 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 10 11:47:30.798245 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application [None req-2a685285-3ba0-4b03-89a8-c016b2d90111 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 10 11:47:30.798245 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:47:30.798245 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:47:30.798245 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:47:30.798245 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:47:30.798245 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:47:30.798245 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:47:30.798245 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:47:30.798245 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:47:30.798245 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:47:30.798245 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:47:30.798245 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:47:30.798245 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 10 11:47:30.798245 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 10 11:47:30.798245 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 10 11:47:30.798245 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 10 11:47:30.800361 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 10 11:47:30.800361 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 10 11:47:30.800361 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 10 11:47:30.800361 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 10 11:47:30.800361 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:47:30.800361 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:47:30.800361 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:47:30.800361 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:47:30.800361 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:47:30.800361 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:47:30.800361 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:47:30.800361 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application with Lock( Oct 10 11:47:30.800361 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:47:30.800361 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:47:30.800361 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:47:30.800361 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:47:30.802370 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:47:30.802370 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:47:30.802370 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:47:30.802370 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:47:30.802370 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 10 11:47:30.802370 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 10 11:47:30.802370 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 10 11:47:30.802370 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 10 11:47:30.802370 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 10 11:47:30.802370 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 10 11:47:30.802370 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 10 11:47:30.802370 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Oct 10 11:47:30.802370 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 390/804] 10.222.0.46 () {56 vars in 1102 bytes} [Mon Oct 10 11:47:30 2022] GET /identity/v3/roles/service => generated 85 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:47:30.802370 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8d0e2d-9dd1-4be8-b15a-f6c030592374 None None] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:30.803916 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8d0e2d-9dd1-4be8-b15a-f6c030592374 None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:30.803916 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8d0e2d-9dd1-4be8-b15a-f6c030592374 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:30.820698 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667b4dfc-e2bd-4d4f-9df6-d2057e284e26 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:30.826062 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.auth.core [None req-2b8d0e2d-9dd1-4be8-b15a-f6c030592374 None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:47:30.827629 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-2b8d0e2d-9dd1-4be8-b15a-f6c030592374 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:47:30.859360 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b0655d-bc0d-4dbd-9f8e-441062843f9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:30.859999 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b0655d-bc0d-4dbd-9f8e-441062843f9f None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:30.860275 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b0655d-bc0d-4dbd-9f8e-441062843f9f None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:30.860546 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b0655d-bc0d-4dbd-9f8e-441062843f9f None admin] PATH_INFO: `/v3/users/swift` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:30.863595 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:30.868856 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b0655d-bc0d-4dbd-9f8e-441062843f9f None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:30.870746 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b0655d-bc0d-4dbd-9f8e-441062843f9f None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:30.887191 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application [None req-70b0655d-bc0d-4dbd-9f8e-441062843f9f None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 10 11:47:30.887191 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:47:30.887191 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:47:30.887191 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:47:30.887191 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:47:30.887191 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:47:30.887191 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:47:30.887191 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:47:30.887191 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:47:30.887191 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:47:30.887191 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:47:30.887191 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:47:30.887191 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 10 11:47:30.887191 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 10 11:47:30.887191 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 10 11:47:30.887191 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 10 11:47:30.888263 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:47:30.888263 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:47:30.888263 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 10 11:47:30.888263 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 10 11:47:30.888263 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 10 11:47:30.888263 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 10 11:47:30.888263 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:47:30.888263 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:47:30.888263 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:47:30.888263 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:47:30.888263 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:47:30.888263 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application with Lock( Oct 10 11:47:30.888263 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:47:30.888263 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:47:30.888263 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:47:30.888263 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:47:30.888263 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:47:30.888263 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:47:30.889550 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:47:30.889550 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:47:30.889550 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 10 11:47:30.889550 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 10 11:47:30.889550 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 10 11:47:30.889550 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 10 11:47:30.889550 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 10 11:47:30.889550 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 10 11:47:30.889550 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 10 11:47:30.889550 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Oct 10 11:47:30.897003 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 415/805] 10.222.0.46 () {56 vars in 1096 bytes} [Mon Oct 10 11:47:30 2022] GET /identity/v3/users/swift => generated 83 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:47:30.897003 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b0655d-bc0d-4dbd-9f8e-441062843f9f None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:30.928023 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 391/806] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 10 11:47:30 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 10 11:47:30.929697 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8d0e2d-9dd1-4be8-b15a-f6c030592374 None None] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:31.000877 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:31.040260 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b0655d-bc0d-4dbd-9f8e-441062843f9f None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:31.046885 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8d0e2d-9dd1-4be8-b15a-f6c030592374 None None] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:31.086241 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627f0fb0-2734-4951-bf40-773c4ea40dec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:31.087064 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627f0fb0-2734-4951-bf40-773c4ea40dec None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:31.087558 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627f0fb0-2734-4951-bf40-773c4ea40dec None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:31.087931 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86adcaf9-cf27-4730-983f-2e8ac8c0041d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:31.088120 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627f0fb0-2734-4951-bf40-773c4ea40dec None admin] PATH_INFO: `/v3/roles` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:31.088559 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86adcaf9-cf27-4730-983f-2e8ac8c0041d None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:31.088841 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86adcaf9-cf27-4730-983f-2e8ac8c0041d None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:31.089082 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627f0fb0-2734-4951-bf40-773c4ea40dec None admin] RBAC: Adding query filter params (name=service) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:47:31.089235 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86adcaf9-cf27-4730-983f-2e8ac8c0041d None admin] PATH_INFO: `/v3/users` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:31.089562 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627f0fb0-2734-4951-bf40-773c4ea40dec None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:31.089855 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86adcaf9-cf27-4730-983f-2e8ac8c0041d None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:47:31.090735 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86adcaf9-cf27-4730-983f-2e8ac8c0041d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:31.091925 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627f0fb0-2734-4951-bf40-773c4ea40dec None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:31.092204 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86adcaf9-cf27-4730-983f-2e8ac8c0041d None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:31.108406 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 416/807] 10.222.0.46 () {56 vars in 1103 bytes} [Mon Oct 10 11:47:30 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:47:31.108689 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627f0fb0-2734-4951-bf40-773c4ea40dec None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:31.123409 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 392/808] 10.222.0.46 () {56 vars in 1135 bytes} [Mon Oct 10 11:47:30 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:47:31.123409 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86adcaf9-cf27-4730-983f-2e8ac8c0041d None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:31.202199 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:31.259887 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86adcaf9-cf27-4730-983f-2e8ac8c0041d None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:31.262945 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627f0fb0-2734-4951-bf40-773c4ea40dec None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:31.283842 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:31.296862 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb3cf1c-d47c-439f-92a5-a5725ade28fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:31.297506 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb3cf1c-d47c-439f-92a5-a5725ade28fd None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:31.307072 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb3cf1c-d47c-439f-92a5-a5725ade28fd None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:31.307072 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb3cf1c-d47c-439f-92a5-a5725ade28fd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:31.307390 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1733bb1-76eb-4690-a078-e05e54b3f945 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:31.307390 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1733bb1-76eb-4690-a078-e05e54b3f945 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:31.307390 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1733bb1-76eb-4690-a078-e05e54b3f945 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:31.307390 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1733bb1-76eb-4690-a078-e05e54b3f945 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:31.309571 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb3cf1c-d47c-439f-92a5-a5725ade28fd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:31.311128 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb3cf1c-d47c-439f-92a5-a5725ade28fd None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:31.320511 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1733bb1-76eb-4690-a078-e05e54b3f945 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 10 11:47:31.326576 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application [None req-c1733bb1-76eb-4690-a078-e05e54b3f945 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 10 11:47:31.326576 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:47:31.326576 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:47:31.326576 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:47:31.326576 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:47:31.326576 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:47:31.326576 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:47:31.326576 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:47:31.326576 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:47:31.326576 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:47:31.326576 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:47:31.326576 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:47:31.326576 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 10 11:47:31.326576 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 10 11:47:31.326576 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 10 11:47:31.326576 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 10 11:47:31.327877 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 10 11:47:31.327877 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 10 11:47:31.327877 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 10 11:47:31.327877 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 10 11:47:31.327877 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:47:31.327877 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:47:31.327877 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:47:31.327877 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:47:31.327877 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:47:31.327877 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:47:31.327877 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:47:31.327877 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application with Lock( Oct 10 11:47:31.327877 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:47:31.327877 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:47:31.327877 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:47:31.327877 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:47:31.328707 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:47:31.328707 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:47:31.328707 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:47:31.328707 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:47:31.328707 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 10 11:47:31.328707 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 10 11:47:31.328707 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 10 11:47:31.328707 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 10 11:47:31.328707 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 10 11:47:31.328707 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 10 11:47:31.328707 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 10 11:47:31.328707 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Oct 10 11:47:31.328707 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 417/809] 10.222.0.46 () {56 vars in 1096 bytes} [Mon Oct 10 11:47:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:47:31.329382 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1733bb1-76eb-4690-a078-e05e54b3f945 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:31.330187 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application [None req-5cb3cf1c-d47c-439f-92a5-a5725ade28fd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:47:31.330187 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:47:31.330187 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:47:31.330187 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:47:31.330187 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:47:31.330187 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:47:31.330187 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:47:31.330187 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:47:31.330187 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:47:31.330187 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:47:31.330187 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:47:31.330187 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:47:31.330187 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 10 11:47:31.330187 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 10 11:47:31.330187 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 10 11:47:31.330187 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 10 11:47:31.331246 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:47:31.331246 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:47:31.331246 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:47:31.331246 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:47:31.331246 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:47:31.331246 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:47:31.331246 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:47:31.331246 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application with Lock( Oct 10 11:47:31.331246 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:47:31.331246 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:47:31.331246 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:47:31.331246 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:47:31.331246 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:47:31.331246 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:47:31.331246 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:47:31.331246 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:47:31.331246 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 10 11:47:31.332120 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 10 11:47:31.332120 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:47:31.332120 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Oct 10 11:47:31.332853 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 393/810] 10.222.0.46 () {56 vars in 1108 bytes} [Mon Oct 10 11:47:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:47:31.334017 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb3cf1c-d47c-439f-92a5-a5725ade28fd None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:31.442116 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1733bb1-76eb-4690-a078-e05e54b3f945 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:31.446396 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb3cf1c-d47c-439f-92a5-a5725ade28fd None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:31.470061 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:31.490534 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0af63c9-e921-4311-88d6-fe8b33f5e685 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:31.491227 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0af63c9-e921-4311-88d6-fe8b33f5e685 None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:31.491510 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0af63c9-e921-4311-88d6-fe8b33f5e685 None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:31.491803 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0af63c9-e921-4311-88d6-fe8b33f5e685 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:31.492413 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0af63c9-e921-4311-88d6-fe8b33f5e685 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:31.492580 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640651bf-ef5b-4527-ab7f-699a0baa10d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:31.493302 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640651bf-ef5b-4527-ab7f-699a0baa10d8 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:31.493498 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640651bf-ef5b-4527-ab7f-699a0baa10d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:31.493749 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640651bf-ef5b-4527-ab7f-699a0baa10d8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:31.494021 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0af63c9-e921-4311-88d6-fe8b33f5e685 None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:31.505104 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application [None req-c0af63c9-e921-4311-88d6-fe8b33f5e685 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 10 11:47:31.505104 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:47:31.505104 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 10 11:47:31.505104 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 10 11:47:31.505104 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 10 11:47:31.505104 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 10 11:47:31.505104 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 10 11:47:31.505104 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 10 11:47:31.505104 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 10 11:47:31.505104 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Oct 10 11:47:31.505104 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 10 11:47:31.505104 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Oct 10 11:47:31.505104 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:47:31.505104 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:47:31.505104 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:47:31.505104 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:47:31.505104 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:47:31.507214 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:47:31.507214 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:47:31.507214 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:47:31.507214 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:47:31.507214 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:47:31.507214 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:47:31.507214 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 10 11:47:31.507214 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 10 11:47:31.507214 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 10 11:47:31.507214 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 10 11:47:31.507214 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:47:31.507214 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:47:31.507214 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:47:31.507214 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:47:31.507214 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:47:31.507214 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:47:31.508884 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:47:31.508884 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application with Lock( Oct 10 11:47:31.508884 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:47:31.508884 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:47:31.508884 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:47:31.508884 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:47:31.508884 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:47:31.508884 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:47:31.508884 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:47:31.508884 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:47:31.508884 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 10 11:47:31.508884 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 10 11:47:31.508884 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 10 11:47:31.508884 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Oct 10 11:47:31.508884 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 394/811] 10.222.0.46 () {56 vars in 1114 bytes} [Mon Oct 10 11:47:31 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:47:31.510475 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0af63c9-e921-4311-88d6-fe8b33f5e685 None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:31.512409 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640651bf-ef5b-4527-ab7f-699a0baa10d8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:31.514306 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640651bf-ef5b-4527-ab7f-699a0baa10d8 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:31.542542 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application [None req-640651bf-ef5b-4527-ab7f-699a0baa10d8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:47:31.542542 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:47:31.542542 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:47:31.542542 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:47:31.542542 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:47:31.542542 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:47:31.542542 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:47:31.542542 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:47:31.542542 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:47:31.542542 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:47:31.542542 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:47:31.542542 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:47:31.542542 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 10 11:47:31.542542 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 10 11:47:31.542542 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 10 11:47:31.542542 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 10 11:47:31.544267 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:47:31.544267 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:47:31.544267 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:47:31.544267 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:47:31.544267 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:47:31.544267 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:47:31.544267 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:47:31.544267 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application with Lock( Oct 10 11:47:31.544267 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:47:31.544267 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:47:31.544267 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:47:31.544267 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:47:31.544267 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:47:31.544267 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:47:31.544267 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:47:31.544267 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:47:31.544267 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 10 11:47:31.545132 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 10 11:47:31.545132 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:47:31.545132 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Oct 10 11:47:31.545132 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 418/812] 10.222.0.46 () {56 vars in 1108 bytes} [Mon Oct 10 11:47:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:47:31.546587 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640651bf-ef5b-4527-ab7f-699a0baa10d8 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:31.559798 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:31.651964 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0af63c9-e921-4311-88d6-fe8b33f5e685 None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:31.670935 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640651bf-ef5b-4527-ab7f-699a0baa10d8 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:31.701261 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a9e412-4b1c-4886-9f0e-697106d262af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:31.701881 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a9e412-4b1c-4886-9f0e-697106d262af None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:31.702175 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a9e412-4b1c-4886-9f0e-697106d262af None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:31.702447 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a9e412-4b1c-4886-9f0e-697106d262af None admin] PATH_INFO: `/v3/roles` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:31.703173 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a9e412-4b1c-4886-9f0e-697106d262af None admin] RBAC: Adding query filter params (name=admin) {{(pid=81676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:47:31.703480 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a9e412-4b1c-4886-9f0e-697106d262af None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:31.704887 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a9e412-4b1c-4886-9f0e-697106d262af None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:31.710197 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e660b947-036a-4972-abf1-cc3266ee1cc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:31.711171 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e660b947-036a-4972-abf1-cc3266ee1cc0 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:31.711511 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e660b947-036a-4972-abf1-cc3266ee1cc0 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:31.711844 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e660b947-036a-4972-abf1-cc3266ee1cc0 None admin] PATH_INFO: `/v3/domains` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:31.712715 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e660b947-036a-4972-abf1-cc3266ee1cc0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:47:31.713082 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e660b947-036a-4972-abf1-cc3266ee1cc0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:31.714237 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 395/813] 10.222.0.46 () {56 vars in 1099 bytes} [Mon Oct 10 11:47:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:47:31.714773 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e660b947-036a-4972-abf1-cc3266ee1cc0 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:31.715172 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a9e412-4b1c-4886-9f0e-697106d262af None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:31.739860 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 419/814] 10.222.0.46 () {56 vars in 1109 bytes} [Mon Oct 10 11:47:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:47:31.741817 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e660b947-036a-4972-abf1-cc3266ee1cc0 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:31.777657 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:31.841013 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:31.845876 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a9e412-4b1c-4886-9f0e-697106d262af None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:31.861910 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e660b947-036a-4972-abf1-cc3266ee1cc0 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:31.896194 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c86f19d-f336-4dc2-99d0-ff2bec562d59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:31.896821 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c86f19d-f336-4dc2-99d0-ff2bec562d59 None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:31.897095 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c86f19d-f336-4dc2-99d0-ff2bec562d59 None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:31.897373 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c86f19d-f336-4dc2-99d0-ff2bec562d59 None admin] PATH_INFO: `/v3/services` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:31.898014 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c86f19d-f336-4dc2-99d0-ff2bec562d59 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:47:31.898316 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c86f19d-f336-4dc2-99d0-ff2bec562d59 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:31.899747 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c86f19d-f336-4dc2-99d0-ff2bec562d59 None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:31.905205 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 396/815] 10.222.0.46 () {56 vars in 1114 bytes} [Mon Oct 10 11:47:31 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:47:31.906030 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c86f19d-f336-4dc2-99d0-ff2bec562d59 None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:31.908283 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa32be67-09f7-4e94-8765-c913b93f0ab1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:31.909341 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa32be67-09f7-4e94-8765-c913b93f0ab1 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:31.909713 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa32be67-09f7-4e94-8765-c913b93f0ab1 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:31.910214 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa32be67-09f7-4e94-8765-c913b93f0ab1 None admin] PATH_INFO: `/v3/domains` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:31.911346 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa32be67-09f7-4e94-8765-c913b93f0ab1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:47:31.911899 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa32be67-09f7-4e94-8765-c913b93f0ab1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:31.914611 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa32be67-09f7-4e94-8765-c913b93f0ab1 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:31.932632 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 420/816] 10.222.0.46 () {56 vars in 1109 bytes} [Mon Oct 10 11:47:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:47:31.933495 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa32be67-09f7-4e94-8765-c913b93f0ab1 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:32.030579 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c86f19d-f336-4dc2-99d0-ff2bec562d59 None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:32.045296 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:32.056924 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa32be67-09f7-4e94-8765-c913b93f0ab1 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:32.080362 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a57c30-ea23-4c89-9bc8-a5140effb202 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:32.081206 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a57c30-ea23-4c89-9bc8-a5140effb202 None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:32.081491 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a57c30-ea23-4c89-9bc8-a5140effb202 None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:32.081827 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a57c30-ea23-4c89-9bc8-a5140effb202 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:32.095912 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a57c30-ea23-4c89-9bc8-a5140effb202 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:32.097416 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a57c30-ea23-4c89-9bc8-a5140effb202 None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:32.112215 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a02677-791a-4754-ba2e-7db8ed0f020d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:32.112858 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a02677-791a-4754-ba2e-7db8ed0f020d None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:32.113138 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a02677-791a-4754-ba2e-7db8ed0f020d None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:32.113411 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a02677-791a-4754-ba2e-7db8ed0f020d None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:32.118994 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application [None req-f1a57c30-ea23-4c89-9bc8-a5140effb202 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:47:32.118994 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:47:32.118994 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:47:32.118994 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:47:32.118994 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:47:32.118994 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:47:32.118994 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:47:32.118994 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:47:32.118994 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:47:32.118994 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:47:32.118994 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:47:32.118994 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:47:32.118994 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 10 11:47:32.118994 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 10 11:47:32.118994 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 10 11:47:32.118994 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 10 11:47:32.121149 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:47:32.121149 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:47:32.121149 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:47:32.121149 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:47:32.121149 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:47:32.121149 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:47:32.121149 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:47:32.121149 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application with Lock( Oct 10 11:47:32.121149 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:47:32.121149 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:47:32.121149 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:47:32.121149 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:47:32.121149 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:47:32.121149 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:47:32.121149 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:47:32.121149 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:47:32.121149 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 10 11:47:32.123307 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 10 11:47:32.123307 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:47:32.123307 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Oct 10 11:47:32.123307 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 397/817] 10.222.0.46 () {56 vars in 1108 bytes} [Mon Oct 10 11:47:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:47:32.123307 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a57c30-ea23-4c89-9bc8-a5140effb202 None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:32.125042 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a02677-791a-4754-ba2e-7db8ed0f020d None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:32.126536 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a02677-791a-4754-ba2e-7db8ed0f020d None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:32.140223 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application [None req-84a02677-791a-4754-ba2e-7db8ed0f020d None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 10 11:47:32.140223 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:47:32.140223 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:47:32.140223 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:47:32.140223 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:47:32.140223 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:47:32.140223 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:47:32.140223 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:47:32.140223 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:47:32.140223 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:47:32.140223 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:47:32.140223 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:47:32.140223 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 10 11:47:32.140223 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 10 11:47:32.140223 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 10 11:47:32.140223 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 10 11:47:32.141650 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:47:32.141650 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:47:32.141650 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 10 11:47:32.141650 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 10 11:47:32.141650 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 10 11:47:32.141650 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 10 11:47:32.141650 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:47:32.141650 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:47:32.141650 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:47:32.141650 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:47:32.141650 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:47:32.141650 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application with Lock( Oct 10 11:47:32.141650 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:47:32.141650 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:47:32.141650 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:47:32.141650 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:47:32.141650 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:47:32.141650 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:47:32.142616 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:47:32.142616 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:47:32.142616 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 10 11:47:32.142616 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 10 11:47:32.142616 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 10 11:47:32.142616 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 10 11:47:32.142616 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 10 11:47:32.142616 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 10 11:47:32.142616 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 10 11:47:32.142616 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Oct 10 11:47:32.142616 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 421/818] 10.222.0.46 () {56 vars in 1117 bytes} [Mon Oct 10 11:47:31 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:47:32.142616 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a02677-791a-4754-ba2e-7db8ed0f020d None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:32.162402 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:32.267999 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a57c30-ea23-4c89-9bc8-a5140effb202 None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:32.282915 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a02677-791a-4754-ba2e-7db8ed0f020d None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:32.310655 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5530e0e9-a69d-45b0-82d9-8cedb07226a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:32.311315 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5530e0e9-a69d-45b0-82d9-8cedb07226a4 None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:32.311589 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5530e0e9-a69d-45b0-82d9-8cedb07226a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:32.311862 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5530e0e9-a69d-45b0-82d9-8cedb07226a4 None admin] PATH_INFO: `/v3/services` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:32.312657 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5530e0e9-a69d-45b0-82d9-8cedb07226a4 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:47:32.312964 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5530e0e9-a69d-45b0-82d9-8cedb07226a4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:32.314799 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5530e0e9-a69d-45b0-82d9-8cedb07226a4 None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:32.320043 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 398/819] 10.222.0.46 () {56 vars in 1114 bytes} [Mon Oct 10 11:47:32 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:47:32.320908 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5530e0e9-a69d-45b0-82d9-8cedb07226a4 None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:32.336908 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6ccd01-7a78-4936-b9dd-34b3577631ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:32.337584 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6ccd01-7a78-4936-b9dd-34b3577631ba None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:32.337861 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6ccd01-7a78-4936-b9dd-34b3577631ba None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:32.338129 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6ccd01-7a78-4936-b9dd-34b3577631ba None admin] PATH_INFO: `/v3/projects/service` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:32.346964 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6ccd01-7a78-4936-b9dd-34b3577631ba None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:32.348526 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6ccd01-7a78-4936-b9dd-34b3577631ba None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:32.361595 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application [None req-4a6ccd01-7a78-4936-b9dd-34b3577631ba None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 10 11:47:32.361595 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:47:32.361595 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:47:32.361595 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:47:32.361595 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:47:32.361595 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:47:32.361595 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:47:32.361595 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:47:32.361595 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:47:32.361595 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:47:32.361595 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:47:32.361595 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:47:32.361595 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 10 11:47:32.361595 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 10 11:47:32.361595 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 10 11:47:32.363412 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 10 11:47:32.363412 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:47:32.363412 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:47:32.363412 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:47:32.363412 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:47:32.363412 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:47:32.363412 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:47:32.363412 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:47:32.363412 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application with Lock( Oct 10 11:47:32.363412 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:47:32.363412 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:47:32.363412 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:47:32.363412 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:47:32.363412 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:47:32.363412 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:47:32.363412 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:47:32.363412 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:47:32.365155 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 10 11:47:32.365155 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 10 11:47:32.365155 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 10 11:47:32.365155 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 10 11:47:32.365155 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 10 11:47:32.365155 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 10 11:47:32.365155 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 10 11:47:32.365155 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Oct 10 11:47:32.365155 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 422/820] 10.222.0.46 () {56 vars in 1111 bytes} [Mon Oct 10 11:47:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:47:32.365155 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6ccd01-7a78-4936-b9dd-34b3577631ba None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:32.383104 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:32.450303 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:32.486346 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5530e0e9-a69d-45b0-82d9-8cedb07226a4 None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:32.516763 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6ccd01-7a78-4936-b9dd-34b3577631ba None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:32.544346 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70010097-3c3b-488b-9ba4-97eed0922798 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:32.544900 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70010097-3c3b-488b-9ba4-97eed0922798 None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:32.545293 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70010097-3c3b-488b-9ba4-97eed0922798 None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:32.545566 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70010097-3c3b-488b-9ba4-97eed0922798 None admin] PATH_INFO: `/v3/domains` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:32.546246 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70010097-3c3b-488b-9ba4-97eed0922798 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:47:32.546543 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70010097-3c3b-488b-9ba4-97eed0922798 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:32.548252 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70010097-3c3b-488b-9ba4-97eed0922798 None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:32.562728 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 399/821] 10.222.0.46 () {56 vars in 1109 bytes} [Mon Oct 10 11:47:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:47:32.563647 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70010097-3c3b-488b-9ba4-97eed0922798 None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:32.570731 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f92d63-1208-40cc-bbe0-2c5cde312a62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:32.571376 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f92d63-1208-40cc-bbe0-2c5cde312a62 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:32.571646 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f92d63-1208-40cc-bbe0-2c5cde312a62 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:32.571920 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f92d63-1208-40cc-bbe0-2c5cde312a62 None admin] PATH_INFO: `/v3/users` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:32.572664 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f92d63-1208-40cc-bbe0-2c5cde312a62 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:47:32.572969 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f92d63-1208-40cc-bbe0-2c5cde312a62 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:32.575713 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f92d63-1208-40cc-bbe0-2c5cde312a62 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:32.597966 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 423/822] 10.222.0.46 () {56 vars in 1149 bytes} [Mon Oct 10 11:47:32 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:47:32.603193 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f92d63-1208-40cc-bbe0-2c5cde312a62 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:32.681970 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:32.721222 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70010097-3c3b-488b-9ba4-97eed0922798 None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:32.724149 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f92d63-1208-40cc-bbe0-2c5cde312a62 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:32.771350 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b70642-db0d-4b71-a66c-84d2bfc1b913 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:32.772202 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae02152-5d00-459b-8f63-9551ad31f356 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:32.772412 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b70642-db0d-4b71-a66c-84d2bfc1b913 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:32.772815 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b70642-db0d-4b71-a66c-84d2bfc1b913 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:32.772926 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae02152-5d00-459b-8f63-9551ad31f356 None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:32.773141 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae02152-5d00-459b-8f63-9551ad31f356 None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:32.773283 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b70642-db0d-4b71-a66c-84d2bfc1b913 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:32.773383 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae02152-5d00-459b-8f63-9551ad31f356 None admin] PATH_INFO: `/v3/projects` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:32.774142 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae02152-5d00-459b-8f63-9551ad31f356 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:47:32.774410 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae02152-5d00-459b-8f63-9551ad31f356 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:32.775932 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae02152-5d00-459b-8f63-9551ad31f356 None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:32.787013 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:32.788414 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b70642-db0d-4b71-a66c-84d2bfc1b913 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:32.789897 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b70642-db0d-4b71-a66c-84d2bfc1b913 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:32.801725 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 400/823] 10.222.0.46 () {56 vars in 1148 bytes} [Mon Oct 10 11:47:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:47:32.801725 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae02152-5d00-459b-8f63-9551ad31f356 None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:32.815554 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application [None req-75b70642-db0d-4b71-a66c-84d2bfc1b913 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 10 11:47:32.815554 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:47:32.815554 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:47:32.815554 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:47:32.815554 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:47:32.815554 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:47:32.815554 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:47:32.815554 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:47:32.815554 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:47:32.815554 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:47:32.815554 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:47:32.815554 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:47:32.815554 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 10 11:47:32.815554 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 10 11:47:32.815554 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 10 11:47:32.815554 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 10 11:47:32.817357 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:47:32.817357 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:47:32.817357 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 10 11:47:32.817357 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 10 11:47:32.817357 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 10 11:47:32.817357 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 10 11:47:32.817357 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:47:32.817357 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:47:32.817357 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:47:32.817357 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:47:32.817357 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:47:32.817357 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application with Lock( Oct 10 11:47:32.817357 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:47:32.817357 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:47:32.817357 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:47:32.817357 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:47:32.817357 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:47:32.817357 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:47:32.818799 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:47:32.818799 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:47:32.818799 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 10 11:47:32.818799 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 10 11:47:32.818799 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 10 11:47:32.818799 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 10 11:47:32.818799 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 10 11:47:32.818799 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 10 11:47:32.818799 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 10 11:47:32.818799 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Oct 10 11:47:32.818799 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 424/824] 10.222.0.46 () {56 vars in 1093 bytes} [Mon Oct 10 11:47:32 2022] GET /identity/v3/users/nova => generated 82 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:47:32.818799 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b70642-db0d-4b71-a66c-84d2bfc1b913 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:32.952293 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b70642-db0d-4b71-a66c-84d2bfc1b913 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:32.957846 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae02152-5d00-459b-8f63-9551ad31f356 None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:32.984378 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:32.993300 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30c57f0-6380-438c-b7fa-943c8b3e0b4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:32.999088 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30c57f0-6380-438c-b7fa-943c8b3e0b4d None admin] REQUEST_METHOD: `PUT` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:32.999088 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30c57f0-6380-438c-b7fa-943c8b3e0b4d None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:32.999088 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30c57f0-6380-438c-b7fa-943c8b3e0b4d None admin] PATH_INFO: `/v3/projects/4ad3772f7b6a47c28b7ba37a9316a3c1/users/15ef3d53ea484237adb0b3d64fdc5480/roles/0d822a0674b6495fac623877a3b38025` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:32.999088 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30c57f0-6380-438c-b7fa-943c8b3e0b4d None admin] RBAC: Authorizing `identity:create_grant(project_id=4ad3772f7b6a47c28b7ba37a9316a3c1, user_id=15ef3d53ea484237adb0b3d64fdc5480, role_id=0d822a0674b6495fac623877a3b38025)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:32.999875 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30c57f0-6380-438c-b7fa-943c8b3e0b4d None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:33.016256 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7100cc48-fb9d-4179-ba41-700649d0f481 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:33.017358 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7100cc48-fb9d-4179-ba41-700649d0f481 None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:33.017865 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7100cc48-fb9d-4179-ba41-700649d0f481 None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:33.018361 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7100cc48-fb9d-4179-ba41-700649d0f481 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:33.037796 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7100cc48-fb9d-4179-ba41-700649d0f481 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:33.040355 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7100cc48-fb9d-4179-ba41-700649d0f481 None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:33.055738 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 425/825] 10.222.0.46 () {58 vars in 1439 bytes} [Mon Oct 10 11:47:32 2022] PUT /identity/v3/projects/4ad3772f7b6a47c28b7ba37a9316a3c1/users/15ef3d53ea484237adb0b3d64fdc5480/roles/0d822a0674b6495fac623877a3b38025 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 11:47:33.056981 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30c57f0-6380-438c-b7fa-943c8b3e0b4d None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:33.064169 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application [None req-7100cc48-fb9d-4179-ba41-700649d0f481 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:47:33.064169 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:47:33.064169 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:47:33.064169 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:47:33.064169 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:47:33.064169 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:47:33.064169 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:47:33.064169 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:47:33.064169 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:47:33.064169 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:47:33.064169 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:47:33.064169 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:47:33.064169 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 10 11:47:33.064169 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 10 11:47:33.064169 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 10 11:47:33.064169 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 10 11:47:33.065456 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:47:33.065456 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:47:33.065456 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:47:33.065456 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:47:33.065456 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:47:33.065456 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:47:33.065456 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:47:33.065456 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application with Lock( Oct 10 11:47:33.065456 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:47:33.065456 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:47:33.065456 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:47:33.065456 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:47:33.065456 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:47:33.065456 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:47:33.065456 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:47:33.065456 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:47:33.065456 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 10 11:47:33.066366 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 10 11:47:33.066366 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:47:33.066366 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Oct 10 11:47:33.066366 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 401/826] 10.222.0.46 () {56 vars in 1108 bytes} [Mon Oct 10 11:47:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:47:33.073334 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7100cc48-fb9d-4179-ba41-700649d0f481 None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:33.169669 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:33.242096 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30c57f0-6380-438c-b7fa-943c8b3e0b4d None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:33.264698 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7100cc48-fb9d-4179-ba41-700649d0f481 None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:33.287413 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b702784-3d60-4a05-bfa4-cde0be9f0031 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:33.288040 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b702784-3d60-4a05-bfa4-cde0be9f0031 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:33.288323 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b702784-3d60-4a05-bfa4-cde0be9f0031 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:33.288588 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b702784-3d60-4a05-bfa4-cde0be9f0031 None admin] PATH_INFO: `/v3/users` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:33.289334 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b702784-3d60-4a05-bfa4-cde0be9f0031 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:47:33.289638 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b702784-3d60-4a05-bfa4-cde0be9f0031 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:33.291145 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b702784-3d60-4a05-bfa4-cde0be9f0031 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:33.309533 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464a3e25-3a90-4714-84b2-dd2b1fc63cff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:33.310119 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464a3e25-3a90-4714-84b2-dd2b1fc63cff None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:33.310416 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464a3e25-3a90-4714-84b2-dd2b1fc63cff None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:33.313640 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464a3e25-3a90-4714-84b2-dd2b1fc63cff None admin] PATH_INFO: `/v3/domains` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:33.313640 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464a3e25-3a90-4714-84b2-dd2b1fc63cff None admin] RBAC: Adding query filter params (name=Default) {{(pid=81676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:47:33.313640 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464a3e25-3a90-4714-84b2-dd2b1fc63cff None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:33.313640 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464a3e25-3a90-4714-84b2-dd2b1fc63cff None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:33.320580 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 426/827] 10.222.0.46 () {56 vars in 1133 bytes} [Mon Oct 10 11:47:33 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:47:33.329020 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b702784-3d60-4a05-bfa4-cde0be9f0031 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:33.336068 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 402/828] 10.222.0.46 () {56 vars in 1109 bytes} [Mon Oct 10 11:47:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 11:47:33.342361 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464a3e25-3a90-4714-84b2-dd2b1fc63cff None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:33.398685 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:33.475369 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b702784-3d60-4a05-bfa4-cde0be9f0031 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:33.498048 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464a3e25-3a90-4714-84b2-dd2b1fc63cff None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:33.516685 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:33.518285 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5971d4f9-e825-4e28-bf77-a36b539afa97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:33.519446 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5971d4f9-e825-4e28-bf77-a36b539afa97 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:33.519906 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5971d4f9-e825-4e28-bf77-a36b539afa97 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:33.520328 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5971d4f9-e825-4e28-bf77-a36b539afa97 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:33.540686 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38869494-4d84-4384-b760-e0811e77fe63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:33.541465 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38869494-4d84-4384-b760-e0811e77fe63 None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:33.541711 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38869494-4d84-4384-b760-e0811e77fe63 None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:33.541988 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38869494-4d84-4384-b760-e0811e77fe63 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:33.551392 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38869494-4d84-4384-b760-e0811e77fe63 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:33.552978 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38869494-4d84-4384-b760-e0811e77fe63 None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:33.566607 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application [None req-38869494-4d84-4384-b760-e0811e77fe63 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 10 11:47:33.566607 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:47:33.566607 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:47:33.566607 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:47:33.566607 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:47:33.566607 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:47:33.566607 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:47:33.566607 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:47:33.566607 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:47:33.566607 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:47:33.566607 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:47:33.566607 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:47:33.566607 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 10 11:47:33.566607 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 10 11:47:33.566607 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 10 11:47:33.567759 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 10 11:47:33.567759 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:47:33.567759 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:47:33.567759 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:47:33.567759 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:47:33.567759 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:47:33.567759 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:47:33.567759 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:47:33.567759 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application with Lock( Oct 10 11:47:33.567759 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:47:33.567759 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:47:33.567759 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:47:33.567759 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:47:33.567759 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:47:33.567759 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:47:33.567759 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:47:33.567759 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:47:33.568832 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 10 11:47:33.568832 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 10 11:47:33.568832 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 10 11:47:33.568832 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 10 11:47:33.568832 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 10 11:47:33.568832 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 10 11:47:33.568832 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 10 11:47:33.568832 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Oct 10 11:47:33.568832 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 403/829] 10.222.0.46 () {56 vars in 1111 bytes} [Mon Oct 10 11:47:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:47:33.573479 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38869494-4d84-4384-b760-e0811e77fe63 None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:33.601754 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5971d4f9-e825-4e28-bf77-a36b539afa97 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:33.603595 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5971d4f9-e825-4e28-bf77-a36b539afa97 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:33.660058 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:33.699172 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38869494-4d84-4384-b760-e0811e77fe63 None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:33.732426 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 427/830] 10.222.0.46 () {58 vars in 1282 bytes} [Mon Oct 10 11:47:33 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 10 11:47:33.738571 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5971d4f9-e825-4e28-bf77-a36b539afa97 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:33.751634 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f08df9-b50f-4ce3-a0f0-c738986d82ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:33.752789 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f08df9-b50f-4ce3-a0f0-c738986d82ea None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:33.753290 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f08df9-b50f-4ce3-a0f0-c738986d82ea None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:33.753981 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f08df9-b50f-4ce3-a0f0-c738986d82ea None admin] PATH_INFO: `/v3/projects` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:33.755078 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f08df9-b50f-4ce3-a0f0-c738986d82ea None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:47:33.755845 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f08df9-b50f-4ce3-a0f0-c738986d82ea None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:33.757596 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f08df9-b50f-4ce3-a0f0-c738986d82ea None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:33.778814 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 404/831] 10.222.0.46 () {56 vars in 1148 bytes} [Mon Oct 10 11:47:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:47:33.782552 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f08df9-b50f-4ce3-a0f0-c738986d82ea None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:33.820687 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:33.865069 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5971d4f9-e825-4e28-bf77-a36b539afa97 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:33.906533 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9781492f-dd9a-489f-aec4-22686fd2c29f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:33.907233 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9781492f-dd9a-489f-aec4-22686fd2c29f None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:33.907517 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9781492f-dd9a-489f-aec4-22686fd2c29f None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:33.907795 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9781492f-dd9a-489f-aec4-22686fd2c29f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:33.920685 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9781492f-dd9a-489f-aec4-22686fd2c29f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:33.922192 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9781492f-dd9a-489f-aec4-22686fd2c29f None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:33.924820 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f08df9-b50f-4ce3-a0f0-c738986d82ea None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:33.930736 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:33.941023 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application [None req-9781492f-dd9a-489f-aec4-22686fd2c29f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:47:33.941023 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:47:33.941023 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:47:33.941023 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:47:33.941023 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:47:33.941023 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:47:33.941023 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:47:33.941023 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:47:33.941023 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:47:33.941023 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:47:33.941023 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:47:33.941023 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:47:33.941023 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 10 11:47:33.941023 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 10 11:47:33.941023 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 10 11:47:33.941023 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 10 11:47:33.948201 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:47:33.948201 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:47:33.948201 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:47:33.948201 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:47:33.948201 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:47:33.948201 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:47:33.948201 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:47:33.948201 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application with Lock( Oct 10 11:47:33.948201 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:47:33.948201 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:47:33.948201 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:47:33.948201 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:47:33.948201 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:47:33.948201 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:47:33.948201 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:47:33.948201 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:47:33.948201 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 10 11:47:33.949112 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 10 11:47:33.949112 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:47:33.949112 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Oct 10 11:47:33.949112 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 428/832] 10.222.0.46 () {56 vars in 1108 bytes} [Mon Oct 10 11:47:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 10 11:47:33.949112 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9781492f-dd9a-489f-aec4-22686fd2c29f None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:33.978304 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361a86cc-c843-4b71-81c5-202c44e92ec2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:33.979045 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361a86cc-c843-4b71-81c5-202c44e92ec2 None admin] REQUEST_METHOD: `PUT` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:33.979331 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361a86cc-c843-4b71-81c5-202c44e92ec2 None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:33.979608 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361a86cc-c843-4b71-81c5-202c44e92ec2 None admin] PATH_INFO: `/v3/projects/4ad3772f7b6a47c28b7ba37a9316a3c1/users/d6430593eecf45d8b891c3b55de4a944/roles/3c35be38787740f4834c728a2ce73b6b` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:33.995902 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361a86cc-c843-4b71-81c5-202c44e92ec2 None admin] RBAC: Authorizing `identity:create_grant(project_id=4ad3772f7b6a47c28b7ba37a9316a3c1, user_id=d6430593eecf45d8b891c3b55de4a944, role_id=3c35be38787740f4834c728a2ce73b6b)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:33.997372 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361a86cc-c843-4b71-81c5-202c44e92ec2 None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:34.065890 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 405/833] 10.222.0.46 () {58 vars in 1439 bytes} [Mon Oct 10 11:47:33 2022] PUT /identity/v3/projects/4ad3772f7b6a47c28b7ba37a9316a3c1/users/d6430593eecf45d8b891c3b55de4a944/roles/3c35be38787740f4834c728a2ce73b6b => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 11:47:34.082652 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9781492f-dd9a-489f-aec4-22686fd2c29f None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:34.130533 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8014b05e-c1bc-48d4-a512-88f129f7eddb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:34.131215 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8014b05e-c1bc-48d4-a512-88f129f7eddb None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:34.131638 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8014b05e-c1bc-48d4-a512-88f129f7eddb None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:34.131926 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8014b05e-c1bc-48d4-a512-88f129f7eddb None admin] PATH_INFO: `/v3/domains` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:34.132707 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8014b05e-c1bc-48d4-a512-88f129f7eddb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:47:34.133134 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8014b05e-c1bc-48d4-a512-88f129f7eddb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:34.135090 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8014b05e-c1bc-48d4-a512-88f129f7eddb None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:34.153238 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 429/834] 10.222.0.46 () {56 vars in 1109 bytes} [Mon Oct 10 11:47:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 11:47:34.159331 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361a86cc-c843-4b71-81c5-202c44e92ec2 None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:34.274611 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:34.302176 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361a86cc-c843-4b71-81c5-202c44e92ec2 None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:34.351967 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d680ed81-56d4-49df-b6d4-266bc0101183 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:34.352539 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d680ed81-56d4-49df-b6d4-266bc0101183 None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:34.352817 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d680ed81-56d4-49df-b6d4-266bc0101183 None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:34.353122 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d680ed81-56d4-49df-b6d4-266bc0101183 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:34.363499 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d680ed81-56d4-49df-b6d4-266bc0101183 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:34.364983 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d680ed81-56d4-49df-b6d4-266bc0101183 None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:34.381132 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application [None req-d680ed81-56d4-49df-b6d4-266bc0101183 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 10 11:47:34.381132 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:47:34.381132 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:47:34.381132 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:47:34.381132 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:47:34.381132 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:47:34.381132 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:47:34.381132 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:47:34.381132 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:47:34.381132 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:47:34.381132 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:47:34.381132 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:47:34.381132 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 10 11:47:34.381132 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 10 11:47:34.381132 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 10 11:47:34.382085 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 10 11:47:34.382085 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:47:34.382085 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:47:34.382085 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:47:34.382085 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:47:34.382085 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:47:34.382085 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:47:34.382085 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:47:34.382085 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application with Lock( Oct 10 11:47:34.382085 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:47:34.382085 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:47:34.382085 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:47:34.382085 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:47:34.382085 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:47:34.382085 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:47:34.382085 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:47:34.382085 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:47:34.383011 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 10 11:47:34.383011 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 10 11:47:34.383011 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 10 11:47:34.383011 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 10 11:47:34.383011 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 10 11:47:34.383011 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 10 11:47:34.383011 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 10 11:47:34.383011 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Oct 10 11:47:34.383568 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 406/835] 10.222.0.46 () {56 vars in 1111 bytes} [Mon Oct 10 11:47:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:47:34.390730 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8014b05e-c1bc-48d4-a512-88f129f7eddb None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:34.458397 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:34.506934 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8014b05e-c1bc-48d4-a512-88f129f7eddb None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:34.544412 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e102c43c-e820-406f-a76e-488f90c3b27d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:34.545419 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e102c43c-e820-406f-a76e-488f90c3b27d None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:34.545419 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e102c43c-e820-406f-a76e-488f90c3b27d None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:34.545681 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e102c43c-e820-406f-a76e-488f90c3b27d None admin] PATH_INFO: `/v3/projects` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:34.546280 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e102c43c-e820-406f-a76e-488f90c3b27d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:47:34.546588 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e102c43c-e820-406f-a76e-488f90c3b27d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:34.548071 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e102c43c-e820-406f-a76e-488f90c3b27d None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:34.562657 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 430/836] 10.222.0.46 () {56 vars in 1148 bytes} [Mon Oct 10 11:47:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:47:34.569284 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d680ed81-56d4-49df-b6d4-266bc0101183 None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:34.754994 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d680ed81-56d4-49df-b6d4-266bc0101183 None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:34.820854 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df46a2ae-6100-4cb6-8a73-6d7c0ce02f4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:34.821894 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df46a2ae-6100-4cb6-8a73-6d7c0ce02f4d None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:34.822396 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df46a2ae-6100-4cb6-8a73-6d7c0ce02f4d None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:34.822932 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df46a2ae-6100-4cb6-8a73-6d7c0ce02f4d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:34.824871 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df46a2ae-6100-4cb6-8a73-6d7c0ce02f4d None admin] RBAC: Adding query filter params (role.id=0d822a0674b6495fac623877a3b38025, scope.project.id=4ad3772f7b6a47c28b7ba37a9316a3c1, user.id=5f130bbb69e546ce99b4e14de81d7e54) {{(pid=81676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:47:34.825434 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df46a2ae-6100-4cb6-8a73-6d7c0ce02f4d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:34.828204 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df46a2ae-6100-4cb6-8a73-6d7c0ce02f4d None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:34.842105 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 407/837] 10.222.0.46 () {56 vars in 1374 bytes} [Mon Oct 10 11:47:34 2022] GET /identity/v3/role_assignments?user.id=5f130bbb69e546ce99b4e14de81d7e54&scope.project.id=4ad3772f7b6a47c28b7ba37a9316a3c1&role.id=0d822a0674b6495fac623877a3b38025 => generated 608 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:47:35.051796 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfed258-d912-49a7-997f-1c8e5bf95900 None None] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:35.052393 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfed258-d912-49a7-997f-1c8e5bf95900 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:35.053084 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfed258-d912-49a7-997f-1c8e5bf95900 None None] PATH_INFO: `/` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:35.056148 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 431/838] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 10 11:47:35 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:47:35.066331 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d63970b-1a80-4ed7-ba9c-db2f35bdd60a None None] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:35.066834 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d63970b-1a80-4ed7-ba9c-db2f35bdd60a None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:35.067294 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d63970b-1a80-4ed7-ba9c-db2f35bdd60a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:35.090840 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:35.099105 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.auth.core [None req-1d63970b-1a80-4ed7-ba9c-db2f35bdd60a None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:47:35.100837 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-1d63970b-1a80-4ed7-ba9c-db2f35bdd60a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:47:35.183060 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 408/839] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 10 11:47:35 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 10 11:47:35.519582 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7f9d36-e668-4902-91fd-1180e20ef8ca None None] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:35.520057 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7f9d36-e668-4902-91fd-1180e20ef8ca None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:35.520500 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7f9d36-e668-4902-91fd-1180e20ef8ca None None] PATH_INFO: `/` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:35.523292 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 432/840] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 10 11:47:35 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:47:35.532354 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62996b8e-e8ea-4f1b-84cb-226235ffba59 None None] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:35.533231 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62996b8e-e8ea-4f1b-84cb-226235ffba59 None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:35.533917 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62996b8e-e8ea-4f1b-84cb-226235ffba59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:35.566120 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.auth.core [None req-62996b8e-e8ea-4f1b-84cb-226235ffba59 None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:47:35.568420 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-62996b8e-e8ea-4f1b-84cb-226235ffba59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:47:35.693476 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 409/841] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 10 11:47:35 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 10 11:47:35.721600 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7f9d36-e668-4902-91fd-1180e20ef8ca None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:35.857389 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7f9d36-e668-4902-91fd-1180e20ef8ca None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:35.880276 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:35.895195 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b09438-0b18-489a-abea-803586d13e2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:35.895824 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b09438-0b18-489a-abea-803586d13e2e None admin] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:35.896105 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b09438-0b18-489a-abea-803586d13e2e None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:35.896379 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b09438-0b18-489a-abea-803586d13e2e None admin] PATH_INFO: `/v3/services` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:35.897049 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b09438-0b18-489a-abea-803586d13e2e None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:35.898422 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b09438-0b18-489a-abea-803586d13e2e None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:36.015739 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 433/842] 10.222.0.46 () {60 vars in 1141 bytes} [Mon Oct 10 11:47:35 2022] POST /identity/v3/services => generated 251 bytes in 295 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 11:47:36.128774 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62996b8e-e8ea-4f1b-84cb-226235ffba59 None None] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:36.255449 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:36.331762 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62996b8e-e8ea-4f1b-84cb-226235ffba59 None None] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:36.332615 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833b024c-7558-414e-b9f1-2b87e59c33c6 None None] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:36.332895 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833b024c-7558-414e-b9f1-2b87e59c33c6 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:36.333136 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833b024c-7558-414e-b9f1-2b87e59c33c6 None None] PATH_INFO: `/` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:36.336648 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 434/843] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 10 11:47:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 10 11:47:36.341549 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b980ec4c-bb86-4d5f-b7ec-41064ea5d3c2 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:36.342013 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b980ec4c-bb86-4d5f-b7ec-41064ea5d3c2 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:36.342492 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b980ec4c-bb86-4d5f-b7ec-41064ea5d3c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:36.374524 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-b980ec4c-bb86-4d5f-b7ec-41064ea5d3c2 None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:47:36.376123 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-b980ec4c-bb86-4d5f-b7ec-41064ea5d3c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:47:36.399729 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce598c6b-faba-490a-9554-a6a9fd99eba9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:36.400890 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce598c6b-faba-490a-9554-a6a9fd99eba9 None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:36.401647 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce598c6b-faba-490a-9554-a6a9fd99eba9 None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:36.402147 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce598c6b-faba-490a-9554-a6a9fd99eba9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:36.423460 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce598c6b-faba-490a-9554-a6a9fd99eba9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 10 11:47:36.434320 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application [None req-ce598c6b-faba-490a-9554-a6a9fd99eba9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 10 11:47:36.434320 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:47:36.434320 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:47:36.434320 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:47:36.434320 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:47:36.434320 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:47:36.434320 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:47:36.434320 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:47:36.434320 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:47:36.434320 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:47:36.434320 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:47:36.434320 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:47:36.434320 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 10 11:47:36.434320 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 10 11:47:36.434320 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 10 11:47:36.434320 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 10 11:47:36.435545 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 10 11:47:36.435545 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 10 11:47:36.435545 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 10 11:47:36.435545 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 10 11:47:36.435545 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:47:36.435545 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:47:36.435545 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:47:36.435545 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:47:36.435545 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:47:36.435545 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:47:36.435545 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:47:36.435545 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application with Lock( Oct 10 11:47:36.435545 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:47:36.435545 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:47:36.435545 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:47:36.435545 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:47:36.436434 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:47:36.436434 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:47:36.436434 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:47:36.436434 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:47:36.436434 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 10 11:47:36.436434 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 10 11:47:36.436434 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 10 11:47:36.436434 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 10 11:47:36.436434 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 10 11:47:36.436434 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 10 11:47:36.436434 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 10 11:47:36.436434 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Oct 10 11:47:36.437121 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 410/844] 10.222.0.46 () {56 vars in 1096 bytes} [Mon Oct 10 11:47:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 308 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 10 11:47:36.446702 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce598c6b-faba-490a-9554-a6a9fd99eba9 None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:36.469348 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 435/845] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 10 11:47:36 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 10 11:47:36.629887 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce598c6b-faba-490a-9554-a6a9fd99eba9 None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:36.672303 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:36.690660 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5030d4-daca-40d6-bce2-6ab3ba2bc3c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:36.691563 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5030d4-daca-40d6-bce2-6ab3ba2bc3c0 None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:36.692009 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5030d4-daca-40d6-bce2-6ab3ba2bc3c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:36.692456 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5030d4-daca-40d6-bce2-6ab3ba2bc3c0 None admin] PATH_INFO: `/v3/roles` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:36.693497 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5030d4-daca-40d6-bce2-6ab3ba2bc3c0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:47:36.693931 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5030d4-daca-40d6-bce2-6ab3ba2bc3c0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:36.695981 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5030d4-daca-40d6-bce2-6ab3ba2bc3c0 None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:36.712061 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 411/846] 10.222.0.46 () {56 vars in 1099 bytes} [Mon Oct 10 11:47:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 11:47:36.720096 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b980ec4c-bb86-4d5f-b7ec-41064ea5d3c2 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:36.774275 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5030d4-daca-40d6-bce2-6ab3ba2bc3c0 None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:36.834994 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b980ec4c-bb86-4d5f-b7ec-41064ea5d3c2 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:36.846385 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:36.876960 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55f9267-d233-44c3-98ad-db3a9a678f4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:36.878097 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55f9267-d233-44c3-98ad-db3a9a678f4b None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:36.878559 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55f9267-d233-44c3-98ad-db3a9a678f4b None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:36.879090 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55f9267-d233-44c3-98ad-db3a9a678f4b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:36.886087 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5030d4-daca-40d6-bce2-6ab3ba2bc3c0 None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:36.893067 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55f9267-d233-44c3-98ad-db3a9a678f4b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:36.898368 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55f9267-d233-44c3-98ad-db3a9a678f4b None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:36.913061 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application [None req-e55f9267-d233-44c3-98ad-db3a9a678f4b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:47:36.913061 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:47:36.913061 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:47:36.913061 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:47:36.913061 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:47:36.913061 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:47:36.913061 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:47:36.913061 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:47:36.913061 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:47:36.913061 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:47:36.913061 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:47:36.913061 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:47:36.913061 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 10 11:47:36.913061 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 10 11:47:36.913061 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 10 11:47:36.913061 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 10 11:47:36.914227 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:47:36.914227 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:47:36.914227 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:47:36.914227 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:47:36.914227 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:47:36.914227 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:47:36.914227 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:47:36.914227 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application with Lock( Oct 10 11:47:36.914227 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:47:36.914227 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:47:36.914227 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:47:36.914227 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:47:36.914227 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:47:36.914227 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:47:36.914227 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:47:36.914227 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:47:36.914227 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 10 11:47:36.915217 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 10 11:47:36.915217 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:47:36.915217 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Oct 10 11:47:36.916996 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 436/847] 10.222.0.46 () {56 vars in 1108 bytes} [Mon Oct 10 11:47:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:47:36.926262 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55f9267-d233-44c3-98ad-db3a9a678f4b None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:36.950827 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c936cad0-6af5-4e95-845a-787ae8d3af4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:36.952321 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c936cad0-6af5-4e95-845a-787ae8d3af4f None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:36.952770 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c936cad0-6af5-4e95-845a-787ae8d3af4f None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:36.953188 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c936cad0-6af5-4e95-845a-787ae8d3af4f None admin] PATH_INFO: `/v3/roles/service` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:36.978342 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c936cad0-6af5-4e95-845a-787ae8d3af4f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 10 11:47:36.991366 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application [None req-c936cad0-6af5-4e95-845a-787ae8d3af4f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 10 11:47:36.991366 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:47:36.991366 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:47:36.991366 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:47:36.991366 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:47:36.991366 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:47:36.991366 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:47:36.991366 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:47:36.991366 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:47:36.991366 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:47:36.991366 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:47:36.991366 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:47:36.991366 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 10 11:47:36.991366 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 10 11:47:36.991366 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 10 11:47:36.991366 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 10 11:47:36.992340 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 10 11:47:36.992340 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 10 11:47:36.992340 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 10 11:47:36.992340 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 10 11:47:36.992340 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:47:36.992340 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:47:36.992340 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:47:36.992340 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:47:36.992340 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:47:36.992340 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:47:36.992340 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:47:36.992340 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application with Lock( Oct 10 11:47:36.992340 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:47:36.992340 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:47:36.992340 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:47:36.992340 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:47:36.993154 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:47:36.993154 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:47:36.993154 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:47:36.993154 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:47:36.993154 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 10 11:47:36.993154 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 10 11:47:36.993154 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 10 11:47:36.993154 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 10 11:47:36.993154 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 10 11:47:36.993154 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 10 11:47:36.993154 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 10 11:47:36.993154 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Oct 10 11:47:36.998744 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 412/848] 10.222.0.46 () {56 vars in 1102 bytes} [Mon Oct 10 11:47:36 2022] GET /identity/v3/roles/service => generated 85 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 10 11:47:37.004701 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c936cad0-6af5-4e95-845a-787ae8d3af4f None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:37.058327 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55f9267-d233-44c3-98ad-db3a9a678f4b None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:37.093479 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:37.097902 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a4bf64-9322-42e1-a468-07c729c61ee3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:37.099346 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a4bf64-9322-42e1-a468-07c729c61ee3 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:37.099861 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a4bf64-9322-42e1-a468-07c729c61ee3 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:37.100343 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a4bf64-9322-42e1-a468-07c729c61ee3 None admin] PATH_INFO: `/v3/domains` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:37.101240 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a4bf64-9322-42e1-a468-07c729c61ee3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:47:37.101798 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a4bf64-9322-42e1-a468-07c729c61ee3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:37.103590 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a4bf64-9322-42e1-a468-07c729c61ee3 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:37.124269 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 437/849] 10.222.0.46 () {56 vars in 1109 bytes} [Mon Oct 10 11:47:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:47:37.126012 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a4bf64-9322-42e1-a468-07c729c61ee3 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:37.183325 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:37.204116 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c936cad0-6af5-4e95-845a-787ae8d3af4f None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:37.247340 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a4bf64-9322-42e1-a468-07c729c61ee3 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:37.267694 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3517270f-a68f-482f-bbf3-799e1fd28b35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:37.268837 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3517270f-a68f-482f-bbf3-799e1fd28b35 None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:37.269392 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3517270f-a68f-482f-bbf3-799e1fd28b35 None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:37.269967 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3517270f-a68f-482f-bbf3-799e1fd28b35 None admin] PATH_INFO: `/v3/roles` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:37.271551 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3517270f-a68f-482f-bbf3-799e1fd28b35 None admin] RBAC: Adding query filter params (name=service) {{(pid=81676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:47:37.272025 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3517270f-a68f-482f-bbf3-799e1fd28b35 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:37.274782 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3517270f-a68f-482f-bbf3-799e1fd28b35 None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:37.289357 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5940aab-6d9b-4c48-aa18-40da19c4e1d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:37.290376 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5940aab-6d9b-4c48-aa18-40da19c4e1d9 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:37.290890 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5940aab-6d9b-4c48-aa18-40da19c4e1d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:37.291345 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5940aab-6d9b-4c48-aa18-40da19c4e1d9 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:37.298091 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 413/850] 10.222.0.46 () {56 vars in 1103 bytes} [Mon Oct 10 11:47:37 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 11:47:37.301369 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3517270f-a68f-482f-bbf3-799e1fd28b35 None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:37.302231 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5940aab-6d9b-4c48-aa18-40da19c4e1d9 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:37.304004 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5940aab-6d9b-4c48-aa18-40da19c4e1d9 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:37.317118 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application [None req-b5940aab-6d9b-4c48-aa18-40da19c4e1d9 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 10 11:47:37.317118 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:47:37.317118 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:47:37.317118 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:47:37.317118 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:47:37.317118 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:47:37.317118 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:47:37.317118 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:47:37.317118 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:47:37.317118 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:47:37.317118 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:47:37.317118 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:47:37.317118 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 10 11:47:37.317118 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 10 11:47:37.317118 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 10 11:47:37.317118 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 10 11:47:37.318123 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:47:37.318123 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:47:37.318123 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 10 11:47:37.318123 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 10 11:47:37.318123 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 10 11:47:37.318123 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 10 11:47:37.318123 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:47:37.318123 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:47:37.318123 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:47:37.318123 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:47:37.318123 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:47:37.318123 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application with Lock( Oct 10 11:47:37.318123 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:47:37.318123 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:47:37.318123 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:47:37.318123 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:47:37.318123 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:47:37.318123 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:47:37.319118 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:47:37.319118 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:47:37.319118 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 10 11:47:37.319118 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 10 11:47:37.319118 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 10 11:47:37.319118 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 10 11:47:37.319118 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 10 11:47:37.319118 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 10 11:47:37.319118 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 10 11:47:37.319118 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Oct 10 11:47:37.327050 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 438/851] 10.222.0.46 () {56 vars in 1096 bytes} [Mon Oct 10 11:47:37 2022] GET /identity/v3/users/swift => generated 83 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 10 11:47:37.329054 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5940aab-6d9b-4c48-aa18-40da19c4e1d9 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:37.464221 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3517270f-a68f-482f-bbf3-799e1fd28b35 None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:37.473227 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:37.473915 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:37.512493 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48200c1-b351-417d-87ff-c2bd5d17df67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:37.513894 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48200c1-b351-417d-87ff-c2bd5d17df67 None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:37.514402 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48200c1-b351-417d-87ff-c2bd5d17df67 None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:37.515126 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48200c1-b351-417d-87ff-c2bd5d17df67 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:37.518762 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5940aab-6d9b-4c48-aa18-40da19c4e1d9 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:37.528820 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48200c1-b351-417d-87ff-c2bd5d17df67 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:37.530549 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48200c1-b351-417d-87ff-c2bd5d17df67 None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:37.550563 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application [None req-e48200c1-b351-417d-87ff-c2bd5d17df67 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:47:37.550563 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:47:37.550563 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:47:37.550563 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:47:37.550563 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:47:37.550563 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:47:37.550563 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:47:37.550563 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:47:37.550563 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:47:37.550563 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:47:37.550563 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:47:37.550563 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:47:37.550563 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 10 11:47:37.550563 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 10 11:47:37.550563 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 10 11:47:37.550563 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 10 11:47:37.551582 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:47:37.551582 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:47:37.551582 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:47:37.551582 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:47:37.551582 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:47:37.551582 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:47:37.551582 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:47:37.551582 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application with Lock( Oct 10 11:47:37.551582 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:47:37.551582 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:47:37.551582 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:47:37.551582 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:47:37.551582 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:47:37.551582 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:47:37.551582 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:47:37.551582 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:47:37.551582 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 10 11:47:37.552532 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 10 11:47:37.552532 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:47:37.552532 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Oct 10 11:47:37.555221 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 414/852] 10.222.0.46 () {56 vars in 1108 bytes} [Mon Oct 10 11:47:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 10 11:47:37.564286 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48200c1-b351-417d-87ff-c2bd5d17df67 None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:37.580844 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2cffaea-9e99-44e8-9762-bf8942f7ff5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:37.581816 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cffaea-9e99-44e8-9762-bf8942f7ff5b None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:37.583087 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cffaea-9e99-44e8-9762-bf8942f7ff5b None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:37.583561 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cffaea-9e99-44e8-9762-bf8942f7ff5b None admin] PATH_INFO: `/v3/users` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:37.584596 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2cffaea-9e99-44e8-9762-bf8942f7ff5b None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:47:37.585128 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2cffaea-9e99-44e8-9762-bf8942f7ff5b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:37.587405 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2cffaea-9e99-44e8-9762-bf8942f7ff5b None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:37.617072 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 439/853] 10.222.0.46 () {56 vars in 1135 bytes} [Mon Oct 10 11:47:37 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 11:47:37.620153 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2cffaea-9e99-44e8-9762-bf8942f7ff5b None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:37.745156 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2cffaea-9e99-44e8-9762-bf8942f7ff5b None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:37.749127 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48200c1-b351-417d-87ff-c2bd5d17df67 None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:37.787822 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c23af7-2c56-470b-af18-3ef2436955df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:37.788530 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c23af7-2c56-470b-af18-3ef2436955df None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:37.788814 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c23af7-2c56-470b-af18-3ef2436955df None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:37.789093 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c23af7-2c56-470b-af18-3ef2436955df None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:37.801772 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:37.821220 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e07a99b-0e34-4a6f-a1f3-50c30400deeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:37.823208 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e07a99b-0e34-4a6f-a1f3-50c30400deeb None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:37.824252 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e07a99b-0e34-4a6f-a1f3-50c30400deeb None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:37.825581 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e07a99b-0e34-4a6f-a1f3-50c30400deeb None admin] PATH_INFO: `/v3/domains` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:37.827499 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e07a99b-0e34-4a6f-a1f3-50c30400deeb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:47:37.828515 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e07a99b-0e34-4a6f-a1f3-50c30400deeb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:37.833125 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e07a99b-0e34-4a6f-a1f3-50c30400deeb None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:37.866971 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 415/854] 10.222.0.46 () {56 vars in 1109 bytes} [Mon Oct 10 11:47:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 11:47:37.874739 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e07a99b-0e34-4a6f-a1f3-50c30400deeb None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:37.884133 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c23af7-2c56-470b-af18-3ef2436955df None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:37.884916 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c23af7-2c56-470b-af18-3ef2436955df None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:37.921541 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:38.044562 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e07a99b-0e34-4a6f-a1f3-50c30400deeb None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:38.049890 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 440/855] 10.222.0.46 () {58 vars in 1282 bytes} [Mon Oct 10 11:47:37 2022] GET /identity/v3/auth/tokens => generated 1262 bytes in 431 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 10 11:47:38.051110 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d350cf56-4817-4be9-a93c-23f784abbe52 None None] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:38.051493 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d350cf56-4817-4be9-a93c-23f784abbe52 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:38.051767 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d350cf56-4817-4be9-a93c-23f784abbe52 None None] PATH_INFO: `/` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:38.053025 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 441/856] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 10 11:47:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:47:38.058394 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d350cf56-4817-4be9-a93c-23f784abbe52 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:38.113879 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c82a30-e359-4811-9b9d-0b89025cced6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:38.115009 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c82a30-e359-4811-9b9d-0b89025cced6 None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:38.115508 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c82a30-e359-4811-9b9d-0b89025cced6 None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:38.116012 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c82a30-e359-4811-9b9d-0b89025cced6 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:38.118141 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:38.130824 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c82a30-e359-4811-9b9d-0b89025cced6 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:38.133590 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c82a30-e359-4811-9b9d-0b89025cced6 None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:38.158162 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application [None req-46c82a30-e359-4811-9b9d-0b89025cced6 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 10 11:47:38.158162 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:47:38.158162 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:47:38.158162 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:47:38.158162 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:47:38.158162 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:47:38.158162 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:47:38.158162 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:47:38.158162 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:47:38.158162 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:47:38.158162 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:47:38.158162 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:47:38.158162 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 10 11:47:38.158162 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 10 11:47:38.158162 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 10 11:47:38.158162 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 10 11:47:38.159415 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:47:38.159415 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:47:38.159415 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 10 11:47:38.159415 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 10 11:47:38.159415 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 10 11:47:38.159415 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 10 11:47:38.159415 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:47:38.159415 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:47:38.159415 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:47:38.159415 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:47:38.159415 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:47:38.159415 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application with Lock( Oct 10 11:47:38.159415 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:47:38.159415 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:47:38.159415 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:47:38.159415 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:47:38.159415 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:47:38.159415 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:47:38.160338 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:47:38.160338 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:47:38.160338 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 10 11:47:38.160338 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 10 11:47:38.160338 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 10 11:47:38.160338 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 10 11:47:38.160338 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 10 11:47:38.160338 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 10 11:47:38.160338 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 10 11:47:38.160338 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Oct 10 11:47:38.160338 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 416/857] 10.222.0.46 () {56 vars in 1117 bytes} [Mon Oct 10 11:47:37 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 10 11:47:38.162450 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70224cdd-294a-424a-8a40-12514cec34fb None None] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:38.162976 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70224cdd-294a-424a-8a40-12514cec34fb None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:38.163482 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70224cdd-294a-424a-8a40-12514cec34fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:38.195866 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.auth.core [None req-70224cdd-294a-424a-8a40-12514cec34fb None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:47:38.197797 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-70224cdd-294a-424a-8a40-12514cec34fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:47:38.231396 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d350cf56-4817-4be9-a93c-23f784abbe52 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:38.300369 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 417/858] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 10 11:47:38 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 10 11:47:38.300369 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70224cdd-294a-424a-8a40-12514cec34fb None None] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:38.314600 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e443493d-4ea7-49f1-b39b-95d3f101b593 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:38.316209 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e443493d-4ea7-49f1-b39b-95d3f101b593 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:38.316921 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e443493d-4ea7-49f1-b39b-95d3f101b593 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:38.317708 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e443493d-4ea7-49f1-b39b-95d3f101b593 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:38.326523 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:38.339032 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e443493d-4ea7-49f1-b39b-95d3f101b593 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:38.342391 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e443493d-4ea7-49f1-b39b-95d3f101b593 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:38.368734 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application [None req-e443493d-4ea7-49f1-b39b-95d3f101b593 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:47:38.368734 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:47:38.368734 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:47:38.368734 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:47:38.368734 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:47:38.368734 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:47:38.368734 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:47:38.368734 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:47:38.368734 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:47:38.368734 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:47:38.368734 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:47:38.368734 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:47:38.368734 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 10 11:47:38.368734 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 10 11:47:38.368734 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 10 11:47:38.368734 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 10 11:47:38.370891 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:47:38.370891 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:47:38.370891 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:47:38.370891 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:47:38.370891 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:47:38.370891 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:47:38.370891 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:47:38.370891 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application with Lock( Oct 10 11:47:38.370891 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:47:38.370891 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:47:38.370891 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:47:38.370891 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:47:38.370891 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:47:38.370891 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:47:38.370891 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:47:38.370891 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:47:38.370891 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 10 11:47:38.372694 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 10 11:47:38.372694 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:47:38.372694 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Oct 10 11:47:38.375417 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 442/859] 10.222.0.46 () {56 vars in 1108 bytes} [Mon Oct 10 11:47:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 317 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:47:38.380132 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e443493d-4ea7-49f1-b39b-95d3f101b593 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:38.522872 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70224cdd-294a-424a-8a40-12514cec34fb None None] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:38.530171 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:38.547790 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e443493d-4ea7-49f1-b39b-95d3f101b593 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:38.565815 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07607f4a-a65e-4d96-834f-acd96c17d3a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:38.566887 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07607f4a-a65e-4d96-834f-acd96c17d3a3 None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:38.567404 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07607f4a-a65e-4d96-834f-acd96c17d3a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:38.567941 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07607f4a-a65e-4d96-834f-acd96c17d3a3 None admin] PATH_INFO: `/v3/users` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:38.569612 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07607f4a-a65e-4d96-834f-acd96c17d3a3 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:47:38.570149 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07607f4a-a65e-4d96-834f-acd96c17d3a3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:38.571882 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07607f4a-a65e-4d96-834f-acd96c17d3a3 None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:38.594094 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 418/860] 10.222.0.46 () {56 vars in 1149 bytes} [Mon Oct 10 11:47:38 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:47:38.606851 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07607f4a-a65e-4d96-834f-acd96c17d3a3 None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:38.613912 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4170b7c4-e770-40be-a7df-2356b9825f19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:38.615280 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4170b7c4-e770-40be-a7df-2356b9825f19 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:38.615935 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4170b7c4-e770-40be-a7df-2356b9825f19 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:38.616624 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4170b7c4-e770-40be-a7df-2356b9825f19 None admin] PATH_INFO: `/v3/domains` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:38.617960 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4170b7c4-e770-40be-a7df-2356b9825f19 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:47:38.618864 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4170b7c4-e770-40be-a7df-2356b9825f19 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:38.622728 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4170b7c4-e770-40be-a7df-2356b9825f19 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:38.652968 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 443/861] 10.222.0.46 () {56 vars in 1109 bytes} [Mon Oct 10 11:47:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 11:47:38.654999 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4170b7c4-e770-40be-a7df-2356b9825f19 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:38.763766 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:38.827206 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07607f4a-a65e-4d96-834f-acd96c17d3a3 None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:38.855461 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4170b7c4-e770-40be-a7df-2356b9825f19 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:38.890978 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bcd37fb-5b91-43db-9142-bd3343b85d65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:38.892029 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcd37fb-5b91-43db-9142-bd3343b85d65 None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:38.892525 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcd37fb-5b91-43db-9142-bd3343b85d65 None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:38.893078 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcd37fb-5b91-43db-9142-bd3343b85d65 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:38.921410 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa9f037-cb6a-4164-ab48-f5cae9d67c95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:38.922639 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa9f037-cb6a-4164-ab48-f5cae9d67c95 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:38.923151 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa9f037-cb6a-4164-ab48-f5cae9d67c95 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:38.923616 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa9f037-cb6a-4164-ab48-f5cae9d67c95 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:38.936847 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa9f037-cb6a-4164-ab48-f5cae9d67c95 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:38.939997 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa9f037-cb6a-4164-ab48-f5cae9d67c95 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:38.963818 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application [None req-baa9f037-cb6a-4164-ab48-f5cae9d67c95 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 10 11:47:38.963818 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:47:38.963818 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:47:38.963818 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:47:38.963818 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:47:38.963818 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:47:38.963818 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:47:38.963818 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:47:38.963818 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:47:38.963818 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:47:38.963818 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:47:38.963818 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:47:38.963818 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 10 11:47:38.963818 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 10 11:47:38.963818 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 10 11:47:38.965329 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 10 11:47:38.965329 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:47:38.965329 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:47:38.965329 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:47:38.965329 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:47:38.965329 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:47:38.965329 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:47:38.965329 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:47:38.965329 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application with Lock( Oct 10 11:47:38.965329 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:47:38.965329 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:47:38.965329 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:47:38.965329 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:47:38.965329 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:47:38.965329 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:47:38.965329 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:47:38.965329 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:47:38.966319 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 10 11:47:38.966319 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 10 11:47:38.966319 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 10 11:47:38.966319 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 10 11:47:38.966319 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 10 11:47:38.966319 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 10 11:47:38.966319 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 10 11:47:38.966319 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Oct 10 11:47:38.966872 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 444/862] 10.222.0.46 () {56 vars in 1111 bytes} [Mon Oct 10 11:47:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 313 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:47:38.968233 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa9f037-cb6a-4164-ab48-f5cae9d67c95 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:38.985213 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:39.008992 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bcd37fb-5b91-43db-9142-bd3343b85d65 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:39.010461 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bcd37fb-5b91-43db-9142-bd3343b85d65 None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:39.103960 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:39.108803 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa9f037-cb6a-4164-ab48-f5cae9d67c95 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:39.155451 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 419/863] 10.222.0.46 () {58 vars in 1282 bytes} [Mon Oct 10 11:47:38 2022] GET /identity/v3/auth/tokens => generated 1262 bytes in 554 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 10 11:47:39.156068 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed481224-5bd1-43ee-ac87-15f55ebc63d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:39.156283 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bcd37fb-5b91-43db-9142-bd3343b85d65 None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:39.157456 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed481224-5bd1-43ee-ac87-15f55ebc63d8 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:39.158244 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed481224-5bd1-43ee-ac87-15f55ebc63d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:39.159042 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed481224-5bd1-43ee-ac87-15f55ebc63d8 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:39.160212 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed481224-5bd1-43ee-ac87-15f55ebc63d8 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:39.162725 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed481224-5bd1-43ee-ac87-15f55ebc63d8 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:39.179764 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application [None req-ed481224-5bd1-43ee-ac87-15f55ebc63d8 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 10 11:47:39.179764 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:47:39.179764 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 10 11:47:39.179764 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 10 11:47:39.179764 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 10 11:47:39.179764 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 10 11:47:39.179764 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 10 11:47:39.179764 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 10 11:47:39.179764 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 10 11:47:39.179764 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Oct 10 11:47:39.179764 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 10 11:47:39.179764 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Oct 10 11:47:39.179764 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:47:39.179764 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:47:39.179764 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:47:39.179764 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:47:39.181600 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:47:39.181600 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:47:39.181600 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:47:39.181600 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:47:39.181600 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:47:39.181600 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:47:39.181600 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:47:39.181600 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 10 11:47:39.181600 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 10 11:47:39.181600 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 10 11:47:39.181600 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 10 11:47:39.181600 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:47:39.181600 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:47:39.181600 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:47:39.181600 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:47:39.181600 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:47:39.181600 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:47:39.183415 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:47:39.183415 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application with Lock( Oct 10 11:47:39.183415 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:47:39.183415 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:47:39.183415 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:47:39.183415 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:47:39.183415 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:47:39.183415 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:47:39.183415 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:47:39.183415 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:47:39.183415 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 10 11:47:39.183415 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 10 11:47:39.183415 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 10 11:47:39.183415 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Oct 10 11:47:39.183415 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 445/864] 10.222.0.46 () {56 vars in 1132 bytes} [Mon Oct 10 11:47:38 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:47:39.185985 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed481224-5bd1-43ee-ac87-15f55ebc63d8 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:39.299353 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bcd37fb-5b91-43db-9142-bd3343b85d65 None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:39.313506 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:39.341402 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422bb3cd-afa1-4229-8fc5-3910c8156dd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:39.342030 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422bb3cd-afa1-4229-8fc5-3910c8156dd0 None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:39.342308 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422bb3cd-afa1-4229-8fc5-3910c8156dd0 None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:39.342582 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422bb3cd-afa1-4229-8fc5-3910c8156dd0 None admin] PATH_INFO: `/v3/projects` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:39.343356 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422bb3cd-afa1-4229-8fc5-3910c8156dd0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:47:39.343662 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422bb3cd-afa1-4229-8fc5-3910c8156dd0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:39.345163 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422bb3cd-afa1-4229-8fc5-3910c8156dd0 None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:39.359042 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 420/865] 10.222.0.46 () {56 vars in 1148 bytes} [Mon Oct 10 11:47:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:47:39.360275 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6362c0-6be0-48a5-85f5-0d94e344173f None None] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:39.360540 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed481224-5bd1-43ee-ac87-15f55ebc63d8 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:39.360696 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6362c0-6be0-48a5-85f5-0d94e344173f None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:39.360849 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6362c0-6be0-48a5-85f5-0d94e344173f None None] PATH_INFO: `/` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:39.366401 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 421/866] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 10 11:47:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:47:39.366401 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6362c0-6be0-48a5-85f5-0d94e344173f None None] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:39.403832 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:39.417501 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e4f747-a419-4478-9ae7-58015a9579c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:39.419189 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e4f747-a419-4478-9ae7-58015a9579c0 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:39.419693 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e4f747-a419-4478-9ae7-58015a9579c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:39.420185 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e4f747-a419-4478-9ae7-58015a9579c0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:39.438113 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e4f747-a419-4478-9ae7-58015a9579c0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:39.440551 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e4f747-a419-4478-9ae7-58015a9579c0 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:39.459141 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application [None req-49e4f747-a419-4478-9ae7-58015a9579c0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:47:39.459141 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:47:39.459141 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:47:39.459141 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:47:39.459141 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:47:39.459141 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:47:39.459141 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:47:39.459141 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:47:39.459141 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:47:39.459141 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:47:39.459141 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:47:39.459141 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:47:39.459141 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 10 11:47:39.459141 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 10 11:47:39.459141 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 10 11:47:39.459141 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 10 11:47:39.460865 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:47:39.460865 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:47:39.460865 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:47:39.460865 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:47:39.460865 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:47:39.460865 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:47:39.460865 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:47:39.460865 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application with Lock( Oct 10 11:47:39.460865 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:47:39.460865 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:47:39.460865 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:47:39.460865 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:47:39.460865 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:47:39.460865 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:47:39.460865 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:47:39.460865 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:47:39.460865 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 10 11:47:39.462305 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 10 11:47:39.462305 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:47:39.462305 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Oct 10 11:47:39.462305 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 446/867] 10.222.0.46 () {56 vars in 1108 bytes} [Mon Oct 10 11:47:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:47:39.462305 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e4f747-a419-4478-9ae7-58015a9579c0 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:39.511602 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6362c0-6be0-48a5-85f5-0d94e344173f None None] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:39.555282 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34f9add-288e-4ff6-b8e5-51a64f00149a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:39.555892 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34f9add-288e-4ff6-b8e5-51a64f00149a None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:39.556170 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34f9add-288e-4ff6-b8e5-51a64f00149a None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:39.556445 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34f9add-288e-4ff6-b8e5-51a64f00149a None admin] PATH_INFO: `/v3/services` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:39.557081 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34f9add-288e-4ff6-b8e5-51a64f00149a None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:47:39.557386 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34f9add-288e-4ff6-b8e5-51a64f00149a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:39.558784 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34f9add-288e-4ff6-b8e5-51a64f00149a None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:39.563896 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 422/868] 10.222.0.46 () {56 vars in 1126 bytes} [Mon Oct 10 11:47:39 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:47:39.565124 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a86d277-2bfc-48f7-b20c-4f1cfab28b78 None None] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:39.565409 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a86d277-2bfc-48f7-b20c-4f1cfab28b78 None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:39.565678 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a86d277-2bfc-48f7-b20c-4f1cfab28b78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:39.584728 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.auth.core [None req-1a86d277-2bfc-48f7-b20c-4f1cfab28b78 None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:47:39.585928 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-1a86d277-2bfc-48f7-b20c-4f1cfab28b78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:47:39.590090 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e4f747-a419-4478-9ae7-58015a9579c0 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:39.618260 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:39.627897 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb662e1-fa6f-4294-a6f8-c09394fdb81c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:39.628541 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb662e1-fa6f-4294-a6f8-c09394fdb81c None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:39.632170 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb662e1-fa6f-4294-a6f8-c09394fdb81c None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:39.632170 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb662e1-fa6f-4294-a6f8-c09394fdb81c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:39.632170 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb662e1-fa6f-4294-a6f8-c09394fdb81c None admin] RBAC: Adding query filter params (role.id=0d822a0674b6495fac623877a3b38025, scope.project.id=4ad3772f7b6a47c28b7ba37a9316a3c1, user.id=15ef3d53ea484237adb0b3d64fdc5480) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:47:39.632170 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb662e1-fa6f-4294-a6f8-c09394fdb81c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:39.632170 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb662e1-fa6f-4294-a6f8-c09394fdb81c None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:39.638688 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 447/869] 10.222.0.46 () {56 vars in 1374 bytes} [Mon Oct 10 11:47:39 2022] GET /identity/v3/role_assignments?user.id=15ef3d53ea484237adb0b3d64fdc5480&scope.project.id=4ad3772f7b6a47c28b7ba37a9316a3c1&role.id=0d822a0674b6495fac623877a3b38025 => generated 608 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:47:39.639660 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb662e1-fa6f-4294-a6f8-c09394fdb81c None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:39.685425 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 423/870] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 10 11:47:39 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 10 11:47:39.692753 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a86d277-2bfc-48f7-b20c-4f1cfab28b78 None None] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:39.751692 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:39.873562 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb662e1-fa6f-4294-a6f8-c09394fdb81c None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:39.918045 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a86d277-2bfc-48f7-b20c-4f1cfab28b78 None None] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:39.931516 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29da515b-a647-4f8f-9d2d-3c5aa62a1c1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:39.932296 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29da515b-a647-4f8f-9d2d-3c5aa62a1c1c None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:39.932420 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29da515b-a647-4f8f-9d2d-3c5aa62a1c1c None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:39.932703 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29da515b-a647-4f8f-9d2d-3c5aa62a1c1c None admin] PATH_INFO: `/v3/domains` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:39.933370 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29da515b-a647-4f8f-9d2d-3c5aa62a1c1c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:47:39.933688 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29da515b-a647-4f8f-9d2d-3c5aa62a1c1c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:39.935165 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29da515b-a647-4f8f-9d2d-3c5aa62a1c1c None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:39.958012 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 448/871] 10.222.0.46 () {56 vars in 1109 bytes} [Mon Oct 10 11:47:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 319 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:47:39.967750 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29da515b-a647-4f8f-9d2d-3c5aa62a1c1c None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:39.985066 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b485da0-e7d8-4bc6-8acb-f1b49bf0bf3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:39.986125 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b485da0-e7d8-4bc6-8acb-f1b49bf0bf3f None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:39.986622 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b485da0-e7d8-4bc6-8acb-f1b49bf0bf3f None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:39.987140 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b485da0-e7d8-4bc6-8acb-f1b49bf0bf3f None admin] PATH_INFO: `/v3/services` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:39.988263 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b485da0-e7d8-4bc6-8acb-f1b49bf0bf3f None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:47:39.988807 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b485da0-e7d8-4bc6-8acb-f1b49bf0bf3f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:39.991440 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b485da0-e7d8-4bc6-8acb-f1b49bf0bf3f None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:40.000651 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 424/872] 10.222.0.46 () {56 vars in 1126 bytes} [Mon Oct 10 11:47:39 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 315 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:47:40.082596 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:40.173041 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29da515b-a647-4f8f-9d2d-3c5aa62a1c1c None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:40.188917 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b485da0-e7d8-4bc6-8acb-f1b49bf0bf3f None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:40.251945 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db29be6-ad22-4b2f-a41c-f206dc4b5bfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:40.253795 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db29be6-ad22-4b2f-a41c-f206dc4b5bfb None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:40.254251 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db29be6-ad22-4b2f-a41c-f206dc4b5bfb None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:40.254719 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db29be6-ad22-4b2f-a41c-f206dc4b5bfb None admin] PATH_INFO: `/v3/projects/service` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:40.269153 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db29be6-ad22-4b2f-a41c-f206dc4b5bfb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:40.271480 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db29be6-ad22-4b2f-a41c-f206dc4b5bfb None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:40.295031 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application [None req-3db29be6-ad22-4b2f-a41c-f206dc4b5bfb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 10 11:47:40.295031 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:47:40.295031 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:47:40.295031 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:47:40.295031 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:47:40.295031 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:47:40.295031 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:47:40.295031 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:47:40.295031 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:47:40.295031 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:47:40.295031 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:47:40.295031 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:47:40.295031 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 10 11:47:40.295031 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 10 11:47:40.295031 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 10 11:47:40.296760 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 10 11:47:40.296760 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:47:40.296760 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:47:40.296760 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:47:40.296760 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:47:40.296760 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:47:40.296760 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:47:40.296760 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:47:40.296760 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application with Lock( Oct 10 11:47:40.296760 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:47:40.296760 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:47:40.296760 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:47:40.296760 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:47:40.296760 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:47:40.296760 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:47:40.296760 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:47:40.296760 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:47:40.299038 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 10 11:47:40.299038 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 10 11:47:40.299038 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 10 11:47:40.299038 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 10 11:47:40.299038 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 10 11:47:40.299038 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 10 11:47:40.299038 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 10 11:47:40.299038 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Oct 10 11:47:40.299038 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 449/873] 10.222.0.46 () {56 vars in 1111 bytes} [Mon Oct 10 11:47:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 331 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:47:40.307750 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db29be6-ad22-4b2f-a41c-f206dc4b5bfb None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:40.380928 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:40.420575 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b485da0-e7d8-4bc6-8acb-f1b49bf0bf3f None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:40.508360 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5ba814-e613-4991-b325-05c213ae3c66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:40.509770 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5ba814-e613-4991-b325-05c213ae3c66 None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:40.510431 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5ba814-e613-4991-b325-05c213ae3c66 None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:40.511354 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5ba814-e613-4991-b325-05c213ae3c66 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:40.512759 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5ba814-e613-4991-b325-05c213ae3c66 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:40.515654 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5ba814-e613-4991-b325-05c213ae3c66 None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:40.530212 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db29be6-ad22-4b2f-a41c-f206dc4b5bfb None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:40.533840 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application [None req-fb5ba814-e613-4991-b325-05c213ae3c66 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 10 11:47:40.533840 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:47:40.533840 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 10 11:47:40.533840 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 10 11:47:40.533840 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 10 11:47:40.533840 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 10 11:47:40.533840 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 10 11:47:40.533840 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 10 11:47:40.533840 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 10 11:47:40.533840 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Oct 10 11:47:40.533840 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 10 11:47:40.533840 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Oct 10 11:47:40.533840 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:47:40.533840 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:47:40.533840 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:47:40.533840 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:47:40.533840 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:47:40.536055 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:47:40.536055 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:47:40.536055 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:47:40.536055 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:47:40.536055 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:47:40.536055 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:47:40.536055 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 10 11:47:40.536055 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 10 11:47:40.536055 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 10 11:47:40.536055 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 10 11:47:40.536055 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:47:40.536055 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:47:40.536055 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:47:40.536055 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:47:40.536055 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:47:40.536055 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:47:40.537726 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:47:40.537726 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application with Lock( Oct 10 11:47:40.537726 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:47:40.537726 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:47:40.537726 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:47:40.537726 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:47:40.537726 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:47:40.537726 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:47:40.537726 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:47:40.537726 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:47:40.537726 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 10 11:47:40.537726 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 10 11:47:40.537726 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 10 11:47:40.537726 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Oct 10 11:47:40.543454 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 425/874] 10.222.0.46 () {56 vars in 1114 bytes} [Mon Oct 10 11:47:40 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 355 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 10 11:47:40.554704 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5ba814-e613-4991-b325-05c213ae3c66 None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:40.555383 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:40.597316 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46205940-ac3b-4310-afe8-a3b83904a5ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:40.598353 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46205940-ac3b-4310-afe8-a3b83904a5ee None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:40.598870 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46205940-ac3b-4310-afe8-a3b83904a5ee None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:40.599346 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46205940-ac3b-4310-afe8-a3b83904a5ee None admin] PATH_INFO: `/v3/projects` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:40.600729 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46205940-ac3b-4310-afe8-a3b83904a5ee None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:47:40.601254 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46205940-ac3b-4310-afe8-a3b83904a5ee None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:40.603977 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46205940-ac3b-4310-afe8-a3b83904a5ee None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:40.635097 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 450/875] 10.222.0.46 () {56 vars in 1148 bytes} [Mon Oct 10 11:47:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 328 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 11:47:40.641181 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46205940-ac3b-4310-afe8-a3b83904a5ee None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:40.753660 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5ba814-e613-4991-b325-05c213ae3c66 None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:40.817072 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c643df-158e-405c-b978-24d4f7c223e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:40.818579 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c643df-158e-405c-b978-24d4f7c223e1 None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:40.819061 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c643df-158e-405c-b978-24d4f7c223e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:40.819624 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c643df-158e-405c-b978-24d4f7c223e1 None admin] PATH_INFO: `/v3/services` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:40.820559 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c643df-158e-405c-b978-24d4f7c223e1 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:47:40.821355 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c643df-158e-405c-b978-24d4f7c223e1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:40.826010 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c643df-158e-405c-b978-24d4f7c223e1 None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:40.834731 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 426/876] 10.222.0.46 () {56 vars in 1114 bytes} [Mon Oct 10 11:47:40 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:47:40.843780 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c643df-158e-405c-b978-24d4f7c223e1 None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:40.855731 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46205940-ac3b-4310-afe8-a3b83904a5ee None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:40.868227 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:40.927463 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33974198-ad06-455b-8ebe-a0c3b501d6df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:40.929302 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33974198-ad06-455b-8ebe-a0c3b501d6df None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:40.930134 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33974198-ad06-455b-8ebe-a0c3b501d6df None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:40.931050 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33974198-ad06-455b-8ebe-a0c3b501d6df None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:40.932868 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33974198-ad06-455b-8ebe-a0c3b501d6df None admin] RBAC: Adding query filter params (role.id=3c35be38787740f4834c728a2ce73b6b, scope.project.id=4ad3772f7b6a47c28b7ba37a9316a3c1, user.id=d6430593eecf45d8b891c3b55de4a944) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:47:40.935045 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33974198-ad06-455b-8ebe-a0c3b501d6df None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:40.938123 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33974198-ad06-455b-8ebe-a0c3b501d6df None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:40.957256 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 451/877] 10.222.0.46 () {56 vars in 1374 bytes} [Mon Oct 10 11:47:40 2022] GET /identity/v3/role_assignments?user.id=d6430593eecf45d8b891c3b55de4a944&scope.project.id=4ad3772f7b6a47c28b7ba37a9316a3c1&role.id=3c35be38787740f4834c728a2ce73b6b => generated 608 bytes in 317 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 11:47:41.031114 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c643df-158e-405c-b978-24d4f7c223e1 None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:41.092516 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7139b501-a6ca-4f20-a885-95c2a167c20f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:41.093501 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7139b501-a6ca-4f20-a885-95c2a167c20f None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:41.093972 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7139b501-a6ca-4f20-a885-95c2a167c20f None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:41.094444 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7139b501-a6ca-4f20-a885-95c2a167c20f None admin] PATH_INFO: `/v3/services` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:41.095590 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7139b501-a6ca-4f20-a885-95c2a167c20f None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:47:41.096178 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7139b501-a6ca-4f20-a885-95c2a167c20f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:41.098718 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7139b501-a6ca-4f20-a885-95c2a167c20f None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:41.108617 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 427/878] 10.222.0.46 () {56 vars in 1114 bytes} [Mon Oct 10 11:47:40 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 11:47:41.117932 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33974198-ad06-455b-8ebe-a0c3b501d6df None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:41.190767 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:41.320491 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33974198-ad06-455b-8ebe-a0c3b501d6df None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:41.389077 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537941a4-f33e-40a8-93b6-62ef8a4b5fd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:41.390419 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537941a4-f33e-40a8-93b6-62ef8a4b5fd6 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:41.391138 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537941a4-f33e-40a8-93b6-62ef8a4b5fd6 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:41.391838 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537941a4-f33e-40a8-93b6-62ef8a4b5fd6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:41.393305 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537941a4-f33e-40a8-93b6-62ef8a4b5fd6 None admin] RBAC: Adding query filter params (interface=public, service_id=d3a4875d6c46440eb3580c5e361e5916, region_id=RegionOne) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:47:41.394079 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537941a4-f33e-40a8-93b6-62ef8a4b5fd6 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:41.396648 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537941a4-f33e-40a8-93b6-62ef8a4b5fd6 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:41.412204 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 452/879] 10.222.0.46 () {56 vars in 1251 bytes} [Mon Oct 10 11:47:41 2022] GET /identity/v3/endpoints?service_id=d3a4875d6c46440eb3580c5e361e5916&interface=public®ion_id=RegionOne => generated 194 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:47:41.421669 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7139b501-a6ca-4f20-a885-95c2a167c20f None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:41.589223 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:41.635370 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7139b501-a6ca-4f20-a885-95c2a167c20f None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:41.698414 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f83c55-7a46-4dc5-b570-c87101768f7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:41.700528 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f83c55-7a46-4dc5-b570-c87101768f7c None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:41.701280 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f83c55-7a46-4dc5-b570-c87101768f7c None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:41.702040 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f83c55-7a46-4dc5-b570-c87101768f7c None admin] PATH_INFO: `/v3/services` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:41.703598 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f83c55-7a46-4dc5-b570-c87101768f7c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:41.706479 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f83c55-7a46-4dc5-b570-c87101768f7c None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:41.718051 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 428/880] 10.222.0.46 () {56 vars in 1087 bytes} [Mon Oct 10 11:47:41 2022] GET /identity/v3/services => generated 1011 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 11:47:42.010035 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cdd841-0cbb-4982-9c28-4cd62fd8a663 None None] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:42.010545 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cdd841-0cbb-4982-9c28-4cd62fd8a663 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:42.011586 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cdd841-0cbb-4982-9c28-4cd62fd8a663 None None] PATH_INFO: `/` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:42.013914 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 453/881] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 10 11:47:42 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:47:42.020021 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93cd3af-1d49-4687-8a8d-23b0140a26eb None None] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:42.020643 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93cd3af-1d49-4687-8a8d-23b0140a26eb None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:42.021129 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93cd3af-1d49-4687-8a8d-23b0140a26eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:42.050385 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.auth.core [None req-b93cd3af-1d49-4687-8a8d-23b0140a26eb None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:47:42.052451 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-b93cd3af-1d49-4687-8a8d-23b0140a26eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:47:42.181456 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 429/882] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 10 11:47:42 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 10 11:47:42.453913 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54cdd841-0cbb-4982-9c28-4cd62fd8a663 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:42.625058 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:42.633327 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54cdd841-0cbb-4982-9c28-4cd62fd8a663 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:42.697597 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4853ff-666b-4e5b-85cc-4ba88a4fecd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:42.698686 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4853ff-666b-4e5b-85cc-4ba88a4fecd1 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:42.700230 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4853ff-666b-4e5b-85cc-4ba88a4fecd1 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:42.700782 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4853ff-666b-4e5b-85cc-4ba88a4fecd1 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:42.702433 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4853ff-666b-4e5b-85cc-4ba88a4fecd1 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:42.705193 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4853ff-666b-4e5b-85cc-4ba88a4fecd1 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:42.723505 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application [None req-4f4853ff-666b-4e5b-85cc-4ba88a4fecd1 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 10 11:47:42.723505 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:47:42.723505 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 10 11:47:42.723505 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 10 11:47:42.723505 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 10 11:47:42.723505 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 10 11:47:42.723505 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 10 11:47:42.723505 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 10 11:47:42.723505 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 10 11:47:42.723505 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Oct 10 11:47:42.723505 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 10 11:47:42.723505 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Oct 10 11:47:42.723505 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:47:42.723505 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:47:42.723505 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:47:42.723505 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:47:42.723505 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:47:42.724533 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:47:42.724533 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:47:42.724533 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:47:42.724533 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:47:42.724533 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:47:42.724533 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:47:42.724533 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 10 11:47:42.724533 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 10 11:47:42.724533 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 10 11:47:42.724533 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 10 11:47:42.724533 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:47:42.724533 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:47:42.724533 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:47:42.724533 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:47:42.724533 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:47:42.724533 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:47:42.725350 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:47:42.725350 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application with Lock( Oct 10 11:47:42.725350 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:47:42.725350 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:47:42.725350 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:47:42.725350 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:47:42.725350 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:47:42.725350 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:47:42.725350 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:47:42.725350 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:47:42.725350 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 10 11:47:42.725350 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 10 11:47:42.725350 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 10 11:47:42.725350 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Oct 10 11:47:42.726072 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 454/883] 10.222.0.46 () {56 vars in 1126 bytes} [Mon Oct 10 11:47:42 2022] GET /identity/v3/services/object-store => generated 93 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:47:42.732859 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93cd3af-1d49-4687-8a8d-23b0140a26eb None None] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:42.852179 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:42.913114 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93cd3af-1d49-4687-8a8d-23b0140a26eb None None] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:42.975010 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff223f4-94c4-4228-9744-8088b5aef151 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:42.976080 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff223f4-94c4-4228-9744-8088b5aef151 None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:42.976567 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff223f4-94c4-4228-9744-8088b5aef151 None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:42.977038 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff223f4-94c4-4228-9744-8088b5aef151 None admin] PATH_INFO: `/v3/services` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:42.978093 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff223f4-94c4-4228-9744-8088b5aef151 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:47:42.978556 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff223f4-94c4-4228-9744-8088b5aef151 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:42.981207 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff223f4-94c4-4228-9744-8088b5aef151 None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:42.990455 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 430/884] 10.222.0.46 () {56 vars in 1122 bytes} [Mon Oct 10 11:47:42 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:47:42.997124 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4853ff-666b-4e5b-85cc-4ba88a4fecd1 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:43.189058 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4853ff-666b-4e5b-85cc-4ba88a4fecd1 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:43.255897 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfe0bdd-9bcd-49de-9294-830243752038 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:43.256851 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfe0bdd-9bcd-49de-9294-830243752038 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:43.257302 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfe0bdd-9bcd-49de-9294-830243752038 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:43.257790 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfe0bdd-9bcd-49de-9294-830243752038 None admin] PATH_INFO: `/v3/services` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:43.258969 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcfe0bdd-9bcd-49de-9294-830243752038 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:47:43.259496 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcfe0bdd-9bcd-49de-9294-830243752038 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:43.262683 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcfe0bdd-9bcd-49de-9294-830243752038 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:43.271858 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 455/885] 10.222.0.46 () {56 vars in 1122 bytes} [Mon Oct 10 11:47:42 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:47:43.729491 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79eb017d-438d-49fa-8c52-4ab58823c185 None None] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:43.730102 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79eb017d-438d-49fa-8c52-4ab58823c185 None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:43.730523 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79eb017d-438d-49fa-8c52-4ab58823c185 None None] PATH_INFO: `/` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:43.733845 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 431/886] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 10 11:47:43 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:47:43.743138 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbe545b-a3ae-4c62-a637-fbe4868f5c75 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:43.744380 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbe545b-a3ae-4c62-a637-fbe4868f5c75 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:43.745137 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbe545b-a3ae-4c62-a637-fbe4868f5c75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:43.776213 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-cbbe545b-a3ae-4c62-a637-fbe4868f5c75 None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:47:43.779356 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-cbbe545b-a3ae-4c62-a637-fbe4868f5c75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:47:43.819133 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:43.914258 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 456/887] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 10 11:47:43 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 10 11:47:44.417594 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79eb017d-438d-49fa-8c52-4ab58823c185 None None] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:44.533512 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb37c08f-a545-4353-8193-c4d2ce215ec1 None None] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:44.534397 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb37c08f-a545-4353-8193-c4d2ce215ec1 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:44.534865 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb37c08f-a545-4353-8193-c4d2ce215ec1 None None] PATH_INFO: `/` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:44.536366 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 457/888] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 10 11:47:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:47:44.542285 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78849ab1-f6d4-465f-b5da-5db9bd6e38ca None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:44.542932 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78849ab1-f6d4-465f-b5da-5db9bd6e38ca None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:44.543371 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78849ab1-f6d4-465f-b5da-5db9bd6e38ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:44.562152 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-78849ab1-f6d4-465f-b5da-5db9bd6e38ca None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:47:44.563796 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-78849ab1-f6d4-465f-b5da-5db9bd6e38ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:47:44.612779 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79eb017d-438d-49fa-8c52-4ab58823c185 None None] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:44.644380 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 458/889] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 10 11:47:44 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 10 11:47:44.647093 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ea6642-5628-485d-b3ea-693a214dea60 None None] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:44.647975 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ea6642-5628-485d-b3ea-693a214dea60 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:44.648638 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ea6642-5628-485d-b3ea-693a214dea60 None None] PATH_INFO: `/` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:44.651577 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 459/890] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 10 11:47:44 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:47:44.659876 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8861e5-0975-40c0-bbcc-52cc2c475ffd None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:44.660941 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8861e5-0975-40c0-bbcc-52cc2c475ffd None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:44.661749 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8861e5-0975-40c0-bbcc-52cc2c475ffd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:44.665966 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b4ea81-e1b0-474e-94a8-abc2dcbe6912 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:44.666588 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b4ea81-e1b0-474e-94a8-abc2dcbe6912 None admin] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:44.666909 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b4ea81-e1b0-474e-94a8-abc2dcbe6912 None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:44.667193 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b4ea81-e1b0-474e-94a8-abc2dcbe6912 None admin] PATH_INFO: `/v3/services` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:44.667975 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b4ea81-e1b0-474e-94a8-abc2dcbe6912 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:44.669281 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b4ea81-e1b0-474e-94a8-abc2dcbe6912 None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:44.672960 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:44.695985 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-2b8861e5-0975-40c0-bbcc-52cc2c475ffd None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:47:44.697956 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-2b8861e5-0975-40c0-bbcc-52cc2c475ffd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:47:44.793340 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 432/891] 10.222.0.46 () {60 vars in 1141 bytes} [Mon Oct 10 11:47:44 2022] POST /identity/v3/services => generated 269 bytes in 377 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 11:47:44.840585 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 460/892] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 10 11:47:44 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 10 11:47:45.013901 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b4ea81-e1b0-474e-94a8-abc2dcbe6912 None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:45.133627 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8861e5-0975-40c0-bbcc-52cc2c475ffd None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:45.149263 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:45.194317 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b4ea81-e1b0-474e-94a8-abc2dcbe6912 None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:45.235881 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9519eed5-c387-4850-8ccf-c7818863ea83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:45.236860 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9519eed5-c387-4850-8ccf-c7818863ea83 None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:45.237347 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9519eed5-c387-4850-8ccf-c7818863ea83 None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:45.237794 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9519eed5-c387-4850-8ccf-c7818863ea83 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:45.238550 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9519eed5-c387-4850-8ccf-c7818863ea83 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:45.240146 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9519eed5-c387-4850-8ccf-c7818863ea83 None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:45.250632 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application [None req-9519eed5-c387-4850-8ccf-c7818863ea83 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 10 11:47:45.250632 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:47:45.250632 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 10 11:47:45.250632 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 10 11:47:45.250632 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 10 11:47:45.250632 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 10 11:47:45.250632 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 10 11:47:45.250632 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 10 11:47:45.250632 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 10 11:47:45.250632 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Oct 10 11:47:45.250632 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 10 11:47:45.250632 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Oct 10 11:47:45.250632 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:47:45.250632 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:47:45.250632 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:47:45.250632 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:47:45.250632 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:47:45.251663 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:47:45.251663 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:47:45.251663 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:47:45.251663 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:47:45.251663 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:47:45.251663 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:47:45.251663 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 10 11:47:45.251663 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 10 11:47:45.251663 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 10 11:47:45.251663 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 10 11:47:45.251663 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:47:45.251663 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:47:45.251663 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:47:45.251663 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:47:45.251663 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:47:45.251663 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:47:45.252470 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:47:45.252470 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application with Lock( Oct 10 11:47:45.252470 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:47:45.252470 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:47:45.252470 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:47:45.252470 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:47:45.252470 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:47:45.252470 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:47:45.252470 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:47:45.252470 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:47:45.252470 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 10 11:47:45.252470 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 10 11:47:45.252470 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 10 11:47:45.252470 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Oct 10 11:47:45.260205 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 433/893] 10.222.0.46 () {56 vars in 1114 bytes} [Mon Oct 10 11:47:45 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:47:45.262286 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9519eed5-c387-4850-8ccf-c7818863ea83 None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:45.265182 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8861e5-0975-40c0-bbcc-52cc2c475ffd None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:45.306739 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a601028-8849-4069-8972-f4dd2988f906 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:45.307493 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a601028-8849-4069-8972-f4dd2988f906 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:45.307853 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a601028-8849-4069-8972-f4dd2988f906 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:45.308677 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a601028-8849-4069-8972-f4dd2988f906 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:45.321006 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a601028-8849-4069-8972-f4dd2988f906 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 10 11:47:45.327254 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application [None req-0a601028-8849-4069-8972-f4dd2988f906 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 10 11:47:45.327254 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:47:45.327254 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:47:45.327254 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:47:45.327254 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:47:45.327254 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:47:45.327254 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:47:45.327254 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:47:45.327254 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:47:45.327254 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:47:45.327254 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:47:45.327254 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:47:45.327254 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 10 11:47:45.327254 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 10 11:47:45.327254 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 10 11:47:45.327254 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 10 11:47:45.329573 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 10 11:47:45.329573 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 10 11:47:45.329573 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 10 11:47:45.329573 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 10 11:47:45.329573 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:47:45.329573 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:47:45.329573 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:47:45.329573 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:47:45.329573 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:47:45.329573 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:47:45.329573 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:47:45.329573 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application with Lock( Oct 10 11:47:45.329573 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:47:45.329573 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:47:45.329573 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:47:45.329573 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:47:45.330997 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:47:45.330997 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:47:45.330997 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:47:45.330997 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:47:45.330997 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 10 11:47:45.330997 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 10 11:47:45.330997 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 10 11:47:45.330997 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 10 11:47:45.330997 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 10 11:47:45.330997 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 10 11:47:45.330997 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 10 11:47:45.330997 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Oct 10 11:47:45.333333 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 461/894] 10.222.0.46 () {56 vars in 1120 bytes} [Mon Oct 10 11:47:45 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 10 11:47:45.334353 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a601028-8849-4069-8972-f4dd2988f906 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:45.364601 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:45.422661 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9519eed5-c387-4850-8ccf-c7818863ea83 None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:45.470081 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:45.491698 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc48b313-0e19-4da0-8702-8de4c7e1ae23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:45.493258 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc48b313-0e19-4da0-8702-8de4c7e1ae23 None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:45.494549 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc48b313-0e19-4da0-8702-8de4c7e1ae23 None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:45.495703 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc48b313-0e19-4da0-8702-8de4c7e1ae23 None admin] PATH_INFO: `/v3/services` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:45.497165 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc48b313-0e19-4da0-8702-8de4c7e1ae23 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:47:45.498779 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc48b313-0e19-4da0-8702-8de4c7e1ae23 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:45.500219 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc48b313-0e19-4da0-8702-8de4c7e1ae23 None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:45.513573 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 434/895] 10.222.0.46 () {56 vars in 1114 bytes} [Mon Oct 10 11:47:45 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 11:47:45.515428 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc48b313-0e19-4da0-8702-8de4c7e1ae23 None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:45.525674 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a601028-8849-4069-8972-f4dd2988f906 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:45.574793 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c336c1be-7617-420b-934e-343ac1ed7a5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:45.575706 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c336c1be-7617-420b-934e-343ac1ed7a5e None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:45.575992 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c336c1be-7617-420b-934e-343ac1ed7a5e None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:45.576269 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c336c1be-7617-420b-934e-343ac1ed7a5e None admin] PATH_INFO: `/v3/roles` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:45.576991 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c336c1be-7617-420b-934e-343ac1ed7a5e None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:47:45.577258 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c336c1be-7617-420b-934e-343ac1ed7a5e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:45.578849 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c336c1be-7617-420b-934e-343ac1ed7a5e None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:45.594758 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 462/896] 10.222.0.46 () {56 vars in 1115 bytes} [Mon Oct 10 11:47:45 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:47:45.594758 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7836a9b3-55bf-4ea6-a31a-a2eb928144cc None None] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:45.594758 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7836a9b3-55bf-4ea6-a31a-a2eb928144cc None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:45.594758 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7836a9b3-55bf-4ea6-a31a-a2eb928144cc None None] PATH_INFO: `/` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:45.595570 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 463/897] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 10 11:47:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:47:45.596398 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7836a9b3-55bf-4ea6-a31a-a2eb928144cc None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:45.668681 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc48b313-0e19-4da0-8702-8de4c7e1ae23 None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:45.704338 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:45.716909 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e618ea3-878b-4f2f-955c-2ddef117dce3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:45.723085 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e618ea3-878b-4f2f-955c-2ddef117dce3 None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:45.723085 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e618ea3-878b-4f2f-955c-2ddef117dce3 None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:45.723085 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e618ea3-878b-4f2f-955c-2ddef117dce3 None admin] PATH_INFO: `/v3/services` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:45.723085 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e618ea3-878b-4f2f-955c-2ddef117dce3 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:47:45.723085 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e618ea3-878b-4f2f-955c-2ddef117dce3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:45.723085 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e618ea3-878b-4f2f-955c-2ddef117dce3 None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:45.728704 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 435/898] 10.222.0.46 () {56 vars in 1114 bytes} [Mon Oct 10 11:47:45 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 11:47:45.729036 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7836a9b3-55bf-4ea6-a31a-a2eb928144cc None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:45.735410 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0059e3c-9418-4f95-86ff-b020f6e0c988 None None] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:45.735961 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0059e3c-9418-4f95-86ff-b020f6e0c988 None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:45.736413 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0059e3c-9418-4f95-86ff-b020f6e0c988 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:45.756149 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.auth.core [None req-c0059e3c-9418-4f95-86ff-b020f6e0c988 None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:47:45.757418 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-c0059e3c-9418-4f95-86ff-b020f6e0c988 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:47:45.775779 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473b07b9-b5d8-4e6c-8668-af7737b22302 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:45.776400 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473b07b9-b5d8-4e6c-8668-af7737b22302 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:45.776754 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473b07b9-b5d8-4e6c-8668-af7737b22302 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:45.777582 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473b07b9-b5d8-4e6c-8668-af7737b22302 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:45.779918 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:45.791026 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473b07b9-b5d8-4e6c-8668-af7737b22302 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:45.793250 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473b07b9-b5d8-4e6c-8668-af7737b22302 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:45.810364 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application [None req-473b07b9-b5d8-4e6c-8668-af7737b22302 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:47:45.810364 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:47:45.810364 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:47:45.810364 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:47:45.810364 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:47:45.810364 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:47:45.810364 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:47:45.810364 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:47:45.810364 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:47:45.810364 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:47:45.810364 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:47:45.810364 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:47:45.810364 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 10 11:47:45.810364 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 10 11:47:45.810364 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 10 11:47:45.810364 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 10 11:47:45.816559 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:47:45.816559 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:47:45.816559 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:47:45.816559 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:47:45.816559 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:47:45.816559 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:47:45.816559 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:47:45.816559 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application with Lock( Oct 10 11:47:45.816559 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:47:45.816559 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:47:45.816559 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:47:45.816559 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:47:45.816559 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:47:45.816559 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:47:45.816559 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:47:45.816559 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:47:45.816559 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 10 11:47:45.817488 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 10 11:47:45.817488 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:47:45.817488 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Oct 10 11:47:45.817488 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 464/899] 10.222.0.46 () {56 vars in 1108 bytes} [Mon Oct 10 11:47:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:47:45.817488 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473b07b9-b5d8-4e6c-8668-af7737b22302 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:45.842710 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 436/900] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 10 11:47:45 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 10 11:47:45.842710 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0059e3c-9418-4f95-86ff-b020f6e0c988 None None] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:45.940289 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473b07b9-b5d8-4e6c-8668-af7737b22302 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:45.996014 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305dda08-335a-4ae3-8f7e-ae8446f7127d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:45.997806 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305dda08-335a-4ae3-8f7e-ae8446f7127d None admin] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:45.998840 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305dda08-335a-4ae3-8f7e-ae8446f7127d None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:45.999346 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305dda08-335a-4ae3-8f7e-ae8446f7127d None admin] PATH_INFO: `/v3/endpoints` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:46.000606 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305dda08-335a-4ae3-8f7e-ae8446f7127d None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:46.002277 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305dda08-335a-4ae3-8f7e-ae8446f7127d None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:46.007763 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0059e3c-9418-4f95-86ff-b020f6e0c988 None None] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:46.032360 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:46.053364 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6fe932-02c7-4fc6-9adc-62239cc31666 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:46.053989 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6fe932-02c7-4fc6-9adc-62239cc31666 None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:46.054266 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6fe932-02c7-4fc6-9adc-62239cc31666 None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:46.054526 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6fe932-02c7-4fc6-9adc-62239cc31666 None admin] PATH_INFO: `/v3/domains` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:46.055225 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6fe932-02c7-4fc6-9adc-62239cc31666 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:47:46.055524 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6fe932-02c7-4fc6-9adc-62239cc31666 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:46.056931 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6fe932-02c7-4fc6-9adc-62239cc31666 None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:46.072673 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 437/901] 10.222.0.46 () {56 vars in 1109 bytes} [Mon Oct 10 11:47:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:47:46.078933 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6fe932-02c7-4fc6-9adc-62239cc31666 None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:46.090327 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 465/902] 10.222.0.46 () {60 vars in 1144 bytes} [Mon Oct 10 11:47:45 2022] POST /identity/v3/endpoints => generated 346 bytes in 278 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 11:47:46.145679 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305dda08-335a-4ae3-8f7e-ae8446f7127d None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:46.271089 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:46.286269 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6fe932-02c7-4fc6-9adc-62239cc31666 None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:46.358412 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0d1370-3cf6-42e4-acb3-c01f06b21f9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:46.359928 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0d1370-3cf6-42e4-acb3-c01f06b21f9d None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:46.360390 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305dda08-335a-4ae3-8f7e-ae8446f7127d None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:46.360935 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0d1370-3cf6-42e4-acb3-c01f06b21f9d None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:46.361675 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0d1370-3cf6-42e4-acb3-c01f06b21f9d None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:46.373603 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0d1370-3cf6-42e4-acb3-c01f06b21f9d None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:46.375174 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0d1370-3cf6-42e4-acb3-c01f06b21f9d None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:46.388297 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application [None req-2d0d1370-3cf6-42e4-acb3-c01f06b21f9d None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 10 11:47:46.388297 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:47:46.388297 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:47:46.388297 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:47:46.388297 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:47:46.388297 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:47:46.388297 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:47:46.388297 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:47:46.388297 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:47:46.388297 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:47:46.388297 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:47:46.388297 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:47:46.388297 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 10 11:47:46.388297 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 10 11:47:46.388297 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 10 11:47:46.388297 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 10 11:47:46.389299 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:47:46.389299 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:47:46.389299 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 10 11:47:46.389299 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 10 11:47:46.389299 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 10 11:47:46.389299 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 10 11:47:46.389299 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:47:46.389299 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:47:46.389299 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:47:46.389299 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:47:46.389299 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:47:46.389299 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application with Lock( Oct 10 11:47:46.389299 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:47:46.389299 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:47:46.389299 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:47:46.389299 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:47:46.389299 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:47:46.389299 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:47:46.390685 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:47:46.390685 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:47:46.390685 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 10 11:47:46.390685 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 10 11:47:46.390685 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 10 11:47:46.390685 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 10 11:47:46.390685 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 10 11:47:46.390685 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 10 11:47:46.390685 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 10 11:47:46.390685 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Oct 10 11:47:46.390685 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 438/903] 10.222.0.46 () {56 vars in 1117 bytes} [Mon Oct 10 11:47:46 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 312 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 10 11:47:46.398694 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0d1370-3cf6-42e4-acb3-c01f06b21f9d None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:46.416132 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f44ce13-d1e9-40f0-847d-c4da6ea357d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:46.417331 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f44ce13-d1e9-40f0-847d-c4da6ea357d9 None admin] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:46.418279 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f44ce13-d1e9-40f0-847d-c4da6ea357d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:46.418556 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f44ce13-d1e9-40f0-847d-c4da6ea357d9 None admin] PATH_INFO: `/v3/services` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:46.419326 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f44ce13-d1e9-40f0-847d-c4da6ea357d9 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:46.421178 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f44ce13-d1e9-40f0-847d-c4da6ea357d9 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:46.473939 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:46.527868 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 466/904] 10.222.0.46 () {60 vars in 1141 bytes} [Mon Oct 10 11:47:46 2022] POST /identity/v3/services => generated 241 bytes in 383 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 10 11:47:46.620954 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0d1370-3cf6-42e4-acb3-c01f06b21f9d None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:46.702620 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b67ce9-9333-4753-8da3-bac8e043a259 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:46.704244 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b67ce9-9333-4753-8da3-bac8e043a259 None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:46.704831 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b67ce9-9333-4753-8da3-bac8e043a259 None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:46.705575 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b67ce9-9333-4753-8da3-bac8e043a259 None admin] PATH_INFO: `/v3/users` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:46.706914 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b67ce9-9333-4753-8da3-bac8e043a259 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:47:46.707769 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b67ce9-9333-4753-8da3-bac8e043a259 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:46.710348 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b67ce9-9333-4753-8da3-bac8e043a259 None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:46.738419 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 439/905] 10.222.0.46 () {56 vars in 1149 bytes} [Mon Oct 10 11:47:46 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 341 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 11:47:46.746201 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f44ce13-d1e9-40f0-847d-c4da6ea357d9 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:46.876567 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:46.879291 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f44ce13-d1e9-40f0-847d-c4da6ea357d9 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:46.944267 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c2c114-2022-48f3-ad13-1b6dd425855b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:46.945318 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c2c114-2022-48f3-ad13-1b6dd425855b None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:46.945804 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c2c114-2022-48f3-ad13-1b6dd425855b None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:46.946221 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c2c114-2022-48f3-ad13-1b6dd425855b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:47.081168 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c2c114-2022-48f3-ad13-1b6dd425855b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:47.083966 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c2c114-2022-48f3-ad13-1b6dd425855b None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:47.187053 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e667a918-5dcd-4150-9ee4-ef304dbfa703 None None] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:47.187807 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e667a918-5dcd-4150-9ee4-ef304dbfa703 None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:47.188304 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e667a918-5dcd-4150-9ee4-ef304dbfa703 None None] PATH_INFO: `/` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:47.192306 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 440/906] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 10 11:47:47 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:47:47.207890 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e19add8-6ba3-4392-b5da-dc3c3c5329b9 None None] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:47.208510 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e19add8-6ba3-4392-b5da-dc3c3c5329b9 None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:47.209106 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e19add8-6ba3-4392-b5da-dc3c3c5329b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:47.234070 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.auth.core [None req-8e19add8-6ba3-4392-b5da-dc3c3c5329b9 None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:47:47.235444 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-8e19add8-6ba3-4392-b5da-dc3c3c5329b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:47:47.256109 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:47.264629 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 467/907] 10.222.0.46 () {58 vars in 1282 bytes} [Mon Oct 10 11:47:46 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 519 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Oct 10 11:47:47.271027 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c2c114-2022-48f3-ad13-1b6dd425855b None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:47.334080 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 441/908] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 10 11:47:47 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 10 11:47:47.479231 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c2c114-2022-48f3-ad13-1b6dd425855b None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:47.546148 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3043cea-6a99-4d8d-9fb3-76312932c56b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:47.547275 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3043cea-6a99-4d8d-9fb3-76312932c56b None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:47.547791 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3043cea-6a99-4d8d-9fb3-76312932c56b None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:47.548292 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3043cea-6a99-4d8d-9fb3-76312932c56b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:47.567961 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3043cea-6a99-4d8d-9fb3-76312932c56b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:47.570933 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3043cea-6a99-4d8d-9fb3-76312932c56b None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:47.602585 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application [None req-b3043cea-6a99-4d8d-9fb3-76312932c56b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:47:47.602585 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:47:47.602585 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:47:47.602585 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:47:47.602585 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:47:47.602585 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:47:47.602585 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:47:47.602585 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:47:47.602585 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:47:47.602585 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:47:47.602585 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:47:47.602585 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:47:47.602585 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 10 11:47:47.602585 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 10 11:47:47.602585 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 10 11:47:47.602585 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 10 11:47:47.603774 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:47:47.603774 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:47:47.603774 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:47:47.603774 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:47:47.603774 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:47:47.603774 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:47:47.603774 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:47:47.603774 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application with Lock( Oct 10 11:47:47.603774 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:47:47.603774 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:47:47.603774 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:47:47.603774 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:47:47.603774 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:47:47.603774 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:47:47.603774 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:47:47.603774 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:47:47.603774 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 10 11:47:47.604666 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 10 11:47:47.604666 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:47:47.604666 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Oct 10 11:47:47.605188 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 468/909] 10.222.0.46 () {56 vars in 1108 bytes} [Mon Oct 10 11:47:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 334 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 10 11:47:47.610542 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e19add8-6ba3-4392-b5da-dc3c3c5329b9 None None] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:47.628896 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3043cea-6a99-4d8d-9fb3-76312932c56b None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:47.632388 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:47.739197 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:47.748986 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e19add8-6ba3-4392-b5da-dc3c3c5329b9 None None] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:47.822091 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f16e54-646d-4c32-9e76-fd73b3e9e681 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:47.823409 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f16e54-646d-4c32-9e76-fd73b3e9e681 None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:47.824472 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f16e54-646d-4c32-9e76-fd73b3e9e681 None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:47.824953 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f16e54-646d-4c32-9e76-fd73b3e9e681 None admin] PATH_INFO: `/v3/domains` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:47.826199 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f16e54-646d-4c32-9e76-fd73b3e9e681 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:47:47.826887 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f16e54-646d-4c32-9e76-fd73b3e9e681 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:47.829596 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f16e54-646d-4c32-9e76-fd73b3e9e681 None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:47.851001 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3043cea-6a99-4d8d-9fb3-76312932c56b None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:47.853389 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 442/910] 10.222.0.46 () {56 vars in 1109 bytes} [Mon Oct 10 11:47:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:47:47.859796 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f16e54-646d-4c32-9e76-fd73b3e9e681 None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:47.919357 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e94ff28-48e1-440e-af1f-8365e25d95f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:47.920791 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e94ff28-48e1-440e-af1f-8365e25d95f8 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:47.921523 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e94ff28-48e1-440e-af1f-8365e25d95f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:47.922214 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e94ff28-48e1-440e-af1f-8365e25d95f8 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:47.923611 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e94ff28-48e1-440e-af1f-8365e25d95f8 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:47.927613 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e94ff28-48e1-440e-af1f-8365e25d95f8 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:47.951557 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application [None req-6e94ff28-48e1-440e-af1f-8365e25d95f8 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 10 11:47:47.951557 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:47:47.951557 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 10 11:47:47.951557 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 10 11:47:47.951557 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 10 11:47:47.951557 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 10 11:47:47.951557 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 10 11:47:47.951557 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 10 11:47:47.951557 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 10 11:47:47.951557 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Oct 10 11:47:47.951557 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 10 11:47:47.951557 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Oct 10 11:47:47.951557 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:47:47.951557 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:47:47.951557 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:47:47.951557 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:47:47.953680 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:47:47.953680 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:47:47.953680 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:47:47.953680 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:47:47.953680 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:47:47.953680 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:47:47.953680 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:47:47.953680 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 10 11:47:47.953680 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 10 11:47:47.953680 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 10 11:47:47.953680 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 10 11:47:47.953680 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:47:47.953680 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:47:47.953680 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:47:47.953680 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:47:47.953680 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:47:47.953680 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:47:47.955446 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:47:47.955446 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application with Lock( Oct 10 11:47:47.955446 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:47:47.955446 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:47:47.955446 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:47:47.955446 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:47:47.955446 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:47:47.955446 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:47:47.955446 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:47:47.955446 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:47:47.955446 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 10 11:47:47.955446 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 10 11:47:47.955446 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 10 11:47:47.955446 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Oct 10 11:47:47.959053 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 469/911] 10.222.0.46 () {56 vars in 1132 bytes} [Mon Oct 10 11:47:47 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 326 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 10 11:47:47.965735 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e94ff28-48e1-440e-af1f-8365e25d95f8 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:47.987071 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:48.014882 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f16e54-646d-4c32-9e76-fd73b3e9e681 None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:48.093844 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62601104-46d1-46ed-883c-d7a988175b67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:48.094987 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62601104-46d1-46ed-883c-d7a988175b67 None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:48.095473 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62601104-46d1-46ed-883c-d7a988175b67 None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:48.095961 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62601104-46d1-46ed-883c-d7a988175b67 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:48.103888 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62601104-46d1-46ed-883c-d7a988175b67 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:48.105767 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62601104-46d1-46ed-883c-d7a988175b67 None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:48.119186 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application [None req-62601104-46d1-46ed-883c-d7a988175b67 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 10 11:47:48.119186 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:47:48.119186 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:47:48.119186 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:47:48.119186 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:47:48.119186 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:47:48.119186 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:47:48.119186 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:47:48.119186 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:47:48.119186 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:47:48.119186 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:47:48.119186 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:47:48.119186 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 10 11:47:48.119186 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 10 11:47:48.119186 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 10 11:47:48.120128 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 10 11:47:48.120128 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:47:48.120128 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:47:48.120128 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:47:48.120128 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:47:48.120128 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:47:48.120128 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:47:48.120128 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:47:48.120128 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application with Lock( Oct 10 11:47:48.120128 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:47:48.120128 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:47:48.120128 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:47:48.120128 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:47:48.120128 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:47:48.120128 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:47:48.120128 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:47:48.120128 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:47:48.121043 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 10 11:47:48.121043 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 10 11:47:48.121043 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 10 11:47:48.121043 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 10 11:47:48.121043 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 10 11:47:48.121043 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 10 11:47:48.121043 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 10 11:47:48.121043 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Oct 10 11:47:48.128304 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 443/912] 10.222.0.46 () {56 vars in 1111 bytes} [Mon Oct 10 11:47:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:47:48.130000 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62601104-46d1-46ed-883c-d7a988175b67 None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:48.152286 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e94ff28-48e1-440e-af1f-8365e25d95f8 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:48.189744 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:48.240352 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a10cd1-22f9-4fb3-9012-1b002ee1b7af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:48.242268 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a10cd1-22f9-4fb3-9012-1b002ee1b7af None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:48.242800 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a10cd1-22f9-4fb3-9012-1b002ee1b7af None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:48.243298 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a10cd1-22f9-4fb3-9012-1b002ee1b7af None admin] PATH_INFO: `/v3/services` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:48.244666 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a10cd1-22f9-4fb3-9012-1b002ee1b7af None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:47:48.245165 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a10cd1-22f9-4fb3-9012-1b002ee1b7af None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:48.248449 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a10cd1-22f9-4fb3-9012-1b002ee1b7af None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:48.266838 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 470/913] 10.222.0.46 () {56 vars in 1126 bytes} [Mon Oct 10 11:47:47 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:47:48.270348 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a10cd1-22f9-4fb3-9012-1b002ee1b7af None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:48.277387 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62601104-46d1-46ed-883c-d7a988175b67 None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:48.343065 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223ef0b3-ec48-42a9-ae26-882c74c3925c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:48.344243 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223ef0b3-ec48-42a9-ae26-882c74c3925c None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:48.344607 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223ef0b3-ec48-42a9-ae26-882c74c3925c None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:48.344928 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223ef0b3-ec48-42a9-ae26-882c74c3925c None admin] PATH_INFO: `/v3/projects` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:48.346316 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223ef0b3-ec48-42a9-ae26-882c74c3925c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:47:48.346986 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223ef0b3-ec48-42a9-ae26-882c74c3925c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:48.351593 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223ef0b3-ec48-42a9-ae26-882c74c3925c None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:48.356899 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:48.377727 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 444/914] 10.222.0.46 () {56 vars in 1148 bytes} [Mon Oct 10 11:47:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 11:47:48.387722 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223ef0b3-ec48-42a9-ae26-882c74c3925c None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:48.409752 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a10cd1-22f9-4fb3-9012-1b002ee1b7af None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:48.485171 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a3f3e0-40fa-4362-8942-4320e91766a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:48.486376 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a3f3e0-40fa-4362-8942-4320e91766a3 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:48.487053 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a3f3e0-40fa-4362-8942-4320e91766a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:48.488211 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a3f3e0-40fa-4362-8942-4320e91766a3 None admin] PATH_INFO: `/v3/services` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:48.489009 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a3f3e0-40fa-4362-8942-4320e91766a3 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:47:48.489639 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a3f3e0-40fa-4362-8942-4320e91766a3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:48.492452 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a3f3e0-40fa-4362-8942-4320e91766a3 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:48.506835 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 471/915] 10.222.0.46 () {56 vars in 1126 bytes} [Mon Oct 10 11:47:48 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 11:47:48.515214 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a3f3e0-40fa-4362-8942-4320e91766a3 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:48.564947 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223ef0b3-ec48-42a9-ae26-882c74c3925c None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:48.600410 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:48.645641 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dcca97c-70d3-4f43-b741-7a12df1d0f57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:48.647053 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcca97c-70d3-4f43-b741-7a12df1d0f57 None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:48.647841 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcca97c-70d3-4f43-b741-7a12df1d0f57 None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:48.648527 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcca97c-70d3-4f43-b741-7a12df1d0f57 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:48.650380 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dcca97c-70d3-4f43-b741-7a12df1d0f57 None admin] RBAC: Adding query filter params (role.id=66c72919d8744554bc2e7c3dcc683e1f, scope.project.id=4ad3772f7b6a47c28b7ba37a9316a3c1, user.id=d6430593eecf45d8b891c3b55de4a944) {{(pid=81676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:47:48.651223 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dcca97c-70d3-4f43-b741-7a12df1d0f57 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:48.656025 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dcca97c-70d3-4f43-b741-7a12df1d0f57 None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:48.675552 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 445/916] 10.222.0.46 () {56 vars in 1374 bytes} [Mon Oct 10 11:47:48 2022] GET /identity/v3/role_assignments?user.id=d6430593eecf45d8b891c3b55de4a944&scope.project.id=4ad3772f7b6a47c28b7ba37a9316a3c1&role.id=66c72919d8744554bc2e7c3dcc683e1f => generated 259 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 11:47:48.730173 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a3f3e0-40fa-4362-8942-4320e91766a3 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:48.807464 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e53fe42-9634-43d3-b23d-8e10b51cf71b None None] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:48.807464 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e53fe42-9634-43d3-b23d-8e10b51cf71b None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:48.807464 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e53fe42-9634-43d3-b23d-8e10b51cf71b None None] PATH_INFO: `/` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:48.809550 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b187bc75-7302-4af0-b9d6-d5d19b7b5df9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:48.809957 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 446/917] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 10 11:47:48 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:47:48.810586 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b187bc75-7302-4af0-b9d6-d5d19b7b5df9 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:48.810915 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b187bc75-7302-4af0-b9d6-d5d19b7b5df9 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:48.811182 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b187bc75-7302-4af0-b9d6-d5d19b7b5df9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:48.811947 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b187bc75-7302-4af0-b9d6-d5d19b7b5df9 None admin] RBAC: Adding query filter params (interface=public, service_id=79880012f01246108cc57ed7736160b5, region_id=RegionOne) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:47:48.812256 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b187bc75-7302-4af0-b9d6-d5d19b7b5df9 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:48.813729 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b187bc75-7302-4af0-b9d6-d5d19b7b5df9 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:48.821303 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b61aec-290d-4acb-aec6-c219b3f1a963 None None] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:48.821937 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b61aec-290d-4acb-aec6-c219b3f1a963 None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:48.822463 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b61aec-290d-4acb-aec6-c219b3f1a963 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:48.822898 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 472/918] 10.222.0.46 () {56 vars in 1251 bytes} [Mon Oct 10 11:47:48 2022] GET /identity/v3/endpoints?service_id=79880012f01246108cc57ed7736160b5&interface=public®ion_id=RegionOne => generated 194 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 11:47:48.831980 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b187bc75-7302-4af0-b9d6-d5d19b7b5df9 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:48.854112 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.auth.core [None req-51b61aec-290d-4acb-aec6-c219b3f1a963 None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:47:48.856387 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-51b61aec-290d-4acb-aec6-c219b3f1a963 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:47:48.965068 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b187bc75-7302-4af0-b9d6-d5d19b7b5df9 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:48.969294 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:48.976255 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:48.994187 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 447/919] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 10 11:47:48 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 10 11:47:49.035301 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ecadca4-a995-4d92-abee-04dfcfbeb04d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:49.036309 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecadca4-a995-4d92-abee-04dfcfbeb04d None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:49.036780 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecadca4-a995-4d92-abee-04dfcfbeb04d None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:49.037270 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecadca4-a995-4d92-abee-04dfcfbeb04d None admin] PATH_INFO: `/v3/services` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:49.038401 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ecadca4-a995-4d92-abee-04dfcfbeb04d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:49.041050 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ecadca4-a995-4d92-abee-04dfcfbeb04d None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:49.054071 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 473/920] 10.222.0.46 () {56 vars in 1087 bytes} [Mon Oct 10 11:47:48 2022] GET /identity/v3/services => generated 1497 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 10 11:47:49.189792 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43810e5d-bbcf-42af-b20f-51f818091efc None None] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:49.190058 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43810e5d-bbcf-42af-b20f-51f818091efc None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:49.190401 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43810e5d-bbcf-42af-b20f-51f818091efc None None] PATH_INFO: `/` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:49.192236 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 448/921] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 10 11:47:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:47:49.198383 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4717202a-e08b-4320-82c2-cf925200b602 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:49.199398 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4717202a-e08b-4320-82c2-cf925200b602 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:49.199552 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4717202a-e08b-4320-82c2-cf925200b602 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:49.219931 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-4717202a-e08b-4320-82c2-cf925200b602 None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:47:49.221159 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-4717202a-e08b-4320-82c2-cf925200b602 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:47:49.284881 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43810e5d-bbcf-42af-b20f-51f818091efc None None] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:49.307512 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 474/922] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 10 11:47:49 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 10 11:47:49.430881 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43810e5d-bbcf-42af-b20f-51f818091efc None None] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:49.474845 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede4f7ce-76d6-4bc5-a5e8-65626249b34d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:49.474845 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede4f7ce-76d6-4bc5-a5e8-65626249b34d None admin] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:49.474845 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede4f7ce-76d6-4bc5-a5e8-65626249b34d None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:49.474845 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede4f7ce-76d6-4bc5-a5e8-65626249b34d None admin] PATH_INFO: `/v3/projects` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:49.475983 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede4f7ce-76d6-4bc5-a5e8-65626249b34d None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:49.475983 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede4f7ce-76d6-4bc5-a5e8-65626249b34d None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:49.476426 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: WARNING keystone.server.flask.common [None req-ede4f7ce-76d6-4bc5-a5e8-65626249b34d None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Oct 10 11:47:49.539816 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.sql.core [None req-ede4f7ce-76d6-4bc5-a5e8-65626249b34d None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Oct 10 11:47:49.539816 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 10 11:47:49.539816 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [parameters: {'id': '9baf5022c4b14b8e9024abe840cc33ec', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Oct 10 11:47:49.539816 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81676) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Oct 10 11:47:49.559382 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application [None req-ede4f7ce-76d6-4bc5-a5e8-65626249b34d None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 9baf5022c4b14b8e9024abe840cc33ec.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 9baf5022c4b14b8e9024abe840cc33ec. Oct 10 11:47:49.559382 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:47:49.559382 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 10 11:47:49.559382 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 10 11:47:49.559382 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 10 11:47:49.559382 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 10 11:47:49.559382 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 10 11:47:49.559382 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application result = self._query(query) Oct 10 11:47:49.559382 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 10 11:47:49.559382 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application conn.query(q) Oct 10 11:47:49.559382 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 10 11:47:49.559382 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 10 11:47:49.559382 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 10 11:47:49.560992 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application result.read() Oct 10 11:47:49.560992 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 10 11:47:49.560992 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 10 11:47:49.560992 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 10 11:47:49.560992 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 10 11:47:49.560992 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 10 11:47:49.560992 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 10 11:47:49.560992 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 10 11:47:49.560992 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 10 11:47:49.560992 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Oct 10 11:47:49.560992 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Oct 10 11:47:49.560992 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Oct 10 11:47:49.560992 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Oct 10 11:47:49.560992 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:47:49.560992 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Oct 10 11:47:49.560992 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return method(*args, **kwargs) Oct 10 11:47:49.560992 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Oct 10 11:47:49.560992 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return project_ref.to_dict() Oct 10 11:47:49.563591 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 10 11:47:49.563591 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application next(self.gen) Oct 10 11:47:49.563591 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Oct 10 11:47:49.563591 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application yield resource Oct 10 11:47:49.563591 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 10 11:47:49.563591 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application next(self.gen) Oct 10 11:47:49.563591 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Oct 10 11:47:49.563591 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application self.session.rollback() Oct 10 11:47:49.563591 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 10 11:47:49.563591 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application self.force_reraise() Oct 10 11:47:49.563591 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 10 11:47:49.563591 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application raise self.value Oct 10 11:47:49.563591 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Oct 10 11:47:49.563591 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Oct 10 11:47:49.563591 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Oct 10 11:47:49.563591 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application session.commit() Oct 10 11:47:49.563591 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Oct 10 11:47:49.566328 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Oct 10 11:47:49.566328 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Oct 10 11:47:49.566328 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application self._prepare_impl() Oct 10 11:47:49.566328 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Oct 10 11:47:49.566328 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application self.session.flush() Oct 10 11:47:49.566328 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Oct 10 11:47:49.566328 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application self._flush(objects) Oct 10 11:47:49.566328 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Oct 10 11:47:49.566328 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Oct 10 11:47:49.566328 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Oct 10 11:47:49.566328 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application compat.raise_( Oct 10 11:47:49.566328 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 10 11:47:49.566328 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application raise exception Oct 10 11:47:49.566328 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Oct 10 11:47:49.566328 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application flush_context.execute() Oct 10 11:47:49.566328 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Oct 10 11:47:49.566328 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application rec.execute(self) Oct 10 11:47:49.573626 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Oct 10 11:47:49.573626 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Oct 10 11:47:49.573626 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Oct 10 11:47:49.573626 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application _emit_insert_statements( Oct 10 11:47:49.573626 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Oct 10 11:47:49.573626 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application c = connection._execute_20( Oct 10 11:47:49.573626 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Oct 10 11:47:49.573626 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Oct 10 11:47:49.573626 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Oct 10 11:47:49.573626 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Oct 10 11:47:49.573626 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Oct 10 11:47:49.573626 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application ret = self._execute_context( Oct 10 11:47:49.573626 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Oct 10 11:47:49.573626 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Oct 10 11:47:49.573626 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Oct 10 11:47:49.576802 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Oct 10 11:47:49.576802 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 10 11:47:49.576802 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application raise exception Oct 10 11:47:49.576802 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 10 11:47:49.576802 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 10 11:47:49.576802 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 10 11:47:49.576802 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 10 11:47:49.576802 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 10 11:47:49.576802 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application result = self._query(query) Oct 10 11:47:49.576802 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 10 11:47:49.576802 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application conn.query(q) Oct 10 11:47:49.576802 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 10 11:47:49.576802 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 10 11:47:49.576802 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 10 11:47:49.576802 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application result.read() Oct 10 11:47:49.576802 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 10 11:47:49.576802 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 10 11:47:49.580217 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 10 11:47:49.580217 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 10 11:47:49.580217 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 10 11:47:49.580217 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 10 11:47:49.580217 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 10 11:47:49.580217 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 10 11:47:49.580217 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Oct 10 11:47:49.580217 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 10 11:47:49.580217 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application [parameters: {'id': '9baf5022c4b14b8e9024abe840cc33ec', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Oct 10 11:47:49.580217 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Oct 10 11:47:49.580217 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Oct 10 11:47:49.580217 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 10 11:47:49.580217 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Oct 10 11:47:49.580217 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:47:49.580217 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Oct 10 11:47:49.583658 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Oct 10 11:47:49.583658 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Oct 10 11:47:49.583658 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Oct 10 11:47:49.583658 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Oct 10 11:47:49.583658 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Oct 10 11:47:49.583658 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 10 11:47:49.583658 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Oct 10 11:47:49.583658 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:47:49.583658 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:47:49.583658 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:47:49.583658 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:47:49.583658 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:47:49.583658 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:47:49.583658 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:47:49.583658 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:47:49.583658 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:47:49.583658 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:47:49.588582 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:47:49.588582 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Oct 10 11:47:49.588582 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Oct 10 11:47:49.588582 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:47:49.588582 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:47:49.588582 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Oct 10 11:47:49.588582 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application raise exception.Conflict( Oct 10 11:47:49.588582 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 9baf5022c4b14b8e9024abe840cc33ec. Oct 10 11:47:49.588582 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Oct 10 11:47:49.588582 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 449/923] 10.222.0.46 () {60 vars in 1140 bytes} [Mon Oct 10 11:47:49 2022] POST /identity/v3/projects => generated 262 bytes in 277 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Oct 10 11:47:49.593331 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4717202a-e08b-4320-82c2-cf925200b602 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:49.749873 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:49.753450 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4717202a-e08b-4320-82c2-cf925200b602 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:49.755541 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede4f7ce-76d6-4bc5-a5e8-65626249b34d None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:49.796940 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec7575a-0ac0-467f-a2c3-0097e1c58a15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:49.797594 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec7575a-0ac0-467f-a2c3-0097e1c58a15 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:49.797874 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec7575a-0ac0-467f-a2c3-0097e1c58a15 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:49.798154 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec7575a-0ac0-467f-a2c3-0097e1c58a15 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:49.807200 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec7575a-0ac0-467f-a2c3-0097e1c58a15 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:49.809277 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec7575a-0ac0-467f-a2c3-0097e1c58a15 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:49.824934 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application [None req-cec7575a-0ac0-467f-a2c3-0097e1c58a15 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 10 11:47:49.824934 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:47:49.824934 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:47:49.824934 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:47:49.824934 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:47:49.824934 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:47:49.824934 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:47:49.824934 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:47:49.824934 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:47:49.824934 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:47:49.824934 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:47:49.824934 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:47:49.824934 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 10 11:47:49.824934 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 10 11:47:49.824934 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 10 11:47:49.826709 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 10 11:47:49.826709 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:47:49.826709 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:47:49.826709 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:47:49.826709 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:47:49.826709 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:47:49.826709 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:47:49.826709 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:47:49.826709 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application with Lock( Oct 10 11:47:49.826709 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:47:49.826709 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:47:49.826709 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:47:49.826709 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:47:49.826709 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:47:49.826709 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:47:49.826709 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:47:49.826709 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:47:49.828462 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 10 11:47:49.828462 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 10 11:47:49.828462 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 10 11:47:49.828462 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 10 11:47:49.828462 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 10 11:47:49.828462 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 10 11:47:49.828462 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 10 11:47:49.828462 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Oct 10 11:47:49.828462 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 475/924] 10.222.0.46 () {56 vars in 1111 bytes} [Mon Oct 10 11:47:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 10 11:47:49.840476 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec7575a-0ac0-467f-a2c3-0097e1c58a15 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:49.846520 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:49.938041 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede4f7ce-76d6-4bc5-a5e8-65626249b34d None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:49.997838 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec7575a-0ac0-467f-a2c3-0097e1c58a15 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:50.001841 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19912d58-949b-4ac5-ba99-2b6cdc0e24c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:50.003786 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19912d58-949b-4ac5-ba99-2b6cdc0e24c0 None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:50.004273 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19912d58-949b-4ac5-ba99-2b6cdc0e24c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:50.004755 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19912d58-949b-4ac5-ba99-2b6cdc0e24c0 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:50.005787 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19912d58-949b-4ac5-ba99-2b6cdc0e24c0 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:50.008707 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19912d58-949b-4ac5-ba99-2b6cdc0e24c0 None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:50.026157 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application [None req-19912d58-949b-4ac5-ba99-2b6cdc0e24c0 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 10 11:47:50.026157 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:47:50.026157 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 10 11:47:50.026157 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 10 11:47:50.026157 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 10 11:47:50.026157 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 10 11:47:50.026157 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 10 11:47:50.026157 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 10 11:47:50.026157 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 10 11:47:50.026157 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Oct 10 11:47:50.026157 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 10 11:47:50.026157 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Oct 10 11:47:50.026157 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:47:50.026157 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:47:50.026157 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:47:50.026157 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:47:50.026157 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:47:50.027511 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:47:50.027511 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:47:50.027511 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:47:50.027511 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:47:50.027511 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:47:50.027511 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:47:50.027511 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 10 11:47:50.027511 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 10 11:47:50.027511 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 10 11:47:50.027511 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 10 11:47:50.027511 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:47:50.027511 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:47:50.027511 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:47:50.027511 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:47:50.027511 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:47:50.027511 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:47:50.028349 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:47:50.028349 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application with Lock( Oct 10 11:47:50.028349 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:47:50.028349 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:47:50.028349 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:47:50.028349 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:47:50.028349 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:47:50.028349 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:47:50.028349 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:47:50.028349 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:47:50.028349 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 10 11:47:50.028349 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 10 11:47:50.028349 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 10 11:47:50.028349 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Oct 10 11:47:50.029080 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 450/925] 10.222.0.46 () {56 vars in 1126 bytes} [Mon Oct 10 11:47:49 2022] GET /identity/v3/services/object-store => generated 93 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 10 11:47:50.040869 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19912d58-949b-4ac5-ba99-2b6cdc0e24c0 None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:50.049676 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8080690b-afe3-4baf-bfd0-7852fabbf80f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:50.050595 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8080690b-afe3-4baf-bfd0-7852fabbf80f None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:50.051104 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8080690b-afe3-4baf-bfd0-7852fabbf80f None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:50.051567 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8080690b-afe3-4baf-bfd0-7852fabbf80f None admin] PATH_INFO: `/v3/projects` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:50.052610 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8080690b-afe3-4baf-bfd0-7852fabbf80f None admin] RBAC: Adding query filter params (name=service) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:47:50.053133 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8080690b-afe3-4baf-bfd0-7852fabbf80f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:50.054813 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8080690b-afe3-4baf-bfd0-7852fabbf80f None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:50.079128 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 476/926] 10.222.0.46 () {56 vars in 1112 bytes} [Mon Oct 10 11:47:49 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 11:47:50.179931 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19912d58-949b-4ac5-ba99-2b6cdc0e24c0 None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:50.195212 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:50.224656 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963c4c57-7780-47b1-ac20-fa88dba18d44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:50.227985 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963c4c57-7780-47b1-ac20-fa88dba18d44 None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:50.227985 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963c4c57-7780-47b1-ac20-fa88dba18d44 None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:50.227985 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963c4c57-7780-47b1-ac20-fa88dba18d44 None admin] PATH_INFO: `/v3/services` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:50.227985 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963c4c57-7780-47b1-ac20-fa88dba18d44 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:47:50.227985 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963c4c57-7780-47b1-ac20-fa88dba18d44 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:50.228630 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963c4c57-7780-47b1-ac20-fa88dba18d44 None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:50.238442 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 451/927] 10.222.0.46 () {56 vars in 1122 bytes} [Mon Oct 10 11:47:50 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:47:50.249559 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8080690b-afe3-4baf-bfd0-7852fabbf80f None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:50.297421 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:50.422218 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8080690b-afe3-4baf-bfd0-7852fabbf80f None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:50.476060 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf9e435-f341-406c-aaed-f641e84d446b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:50.477087 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf9e435-f341-406c-aaed-f641e84d446b None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:50.477610 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf9e435-f341-406c-aaed-f641e84d446b None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:50.478095 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf9e435-f341-406c-aaed-f641e84d446b None admin] PATH_INFO: `/v3/services` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:50.479067 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf9e435-f341-406c-aaed-f641e84d446b None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:47:50.479590 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf9e435-f341-406c-aaed-f641e84d446b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:50.481784 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf9e435-f341-406c-aaed-f641e84d446b None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:50.488766 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 477/928] 10.222.0.46 () {56 vars in 1122 bytes} [Mon Oct 10 11:47:50 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 11:47:50.500349 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963c4c57-7780-47b1-ac20-fa88dba18d44 None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:50.678486 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963c4c57-7780-47b1-ac20-fa88dba18d44 None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:50.723060 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e85b55-c065-4eb9-86cf-ace51d3cadb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:50.723697 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e85b55-c065-4eb9-86cf-ace51d3cadb1 None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:50.723980 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e85b55-c065-4eb9-86cf-ace51d3cadb1 None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:50.724258 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e85b55-c065-4eb9-86cf-ace51d3cadb1 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:50.725008 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e85b55-c065-4eb9-86cf-ace51d3cadb1 None admin] RBAC: Adding query filter params (interface=public, service_id=bebea60cce1243768657f317c2f37e04, region_id=RegionOne) {{(pid=81676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:47:50.725321 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e85b55-c065-4eb9-86cf-ace51d3cadb1 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:50.726770 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e85b55-c065-4eb9-86cf-ace51d3cadb1 None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:50.733316 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 452/929] 10.222.0.46 () {56 vars in 1251 bytes} [Mon Oct 10 11:47:50 2022] GET /identity/v3/endpoints?service_id=bebea60cce1243768657f317c2f37e04&interface=public®ion_id=RegionOne => generated 194 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:47:50.743028 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf9e435-f341-406c-aaed-f641e84d446b None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:50.845594 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:50.864719 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf9e435-f341-406c-aaed-f641e84d446b None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:50.906509 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb52ca5e-6b8e-40f4-ada6-db7a1544441a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:50.907181 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb52ca5e-6b8e-40f4-ada6-db7a1544441a None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:50.907466 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb52ca5e-6b8e-40f4-ada6-db7a1544441a None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:50.907742 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb52ca5e-6b8e-40f4-ada6-db7a1544441a None admin] PATH_INFO: `/v3/services` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:50.908377 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb52ca5e-6b8e-40f4-ada6-db7a1544441a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:50.909796 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb52ca5e-6b8e-40f4-ada6-db7a1544441a None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:50.917053 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 478/930] 10.222.0.46 () {56 vars in 1087 bytes} [Mon Oct 10 11:47:50 2022] GET /identity/v3/services => generated 1497 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 11:47:51.276626 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35a1166-fd3c-4b86-9e55-0774767d751c None None] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:51.277593 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35a1166-fd3c-4b86-9e55-0774767d751c None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:51.277745 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35a1166-fd3c-4b86-9e55-0774767d751c None None] PATH_INFO: `/` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:51.279527 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 453/931] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 10 11:47:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:47:51.285656 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd31821-a38b-4730-8cd5-b32847ba5f18 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:51.286503 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd31821-a38b-4730-8cd5-b32847ba5f18 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:51.286503 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd31821-a38b-4730-8cd5-b32847ba5f18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:51.306967 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-6fd31821-a38b-4730-8cd5-b32847ba5f18 None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:47:51.308774 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-6fd31821-a38b-4730-8cd5-b32847ba5f18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:47:51.383794 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 479/932] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 10 11:47:51 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 10 11:47:51.652699 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35a1166-fd3c-4b86-9e55-0774767d751c None None] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:51.700588 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:51.771392 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35a1166-fd3c-4b86-9e55-0774767d751c None None] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:51.835209 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d8c78f-eaf5-489a-99a2-b0d2c9759573 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:51.836793 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d8c78f-eaf5-489a-99a2-b0d2c9759573 None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:51.837587 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d8c78f-eaf5-489a-99a2-b0d2c9759573 None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:51.838393 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d8c78f-eaf5-489a-99a2-b0d2c9759573 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:51.860136 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d8c78f-eaf5-489a-99a2-b0d2c9759573 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 10 11:47:51.871277 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application [None req-e2d8c78f-eaf5-489a-99a2-b0d2c9759573 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 10 11:47:51.871277 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:47:51.871277 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:47:51.871277 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:47:51.871277 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:47:51.871277 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:47:51.871277 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:47:51.871277 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:47:51.871277 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:47:51.871277 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:47:51.871277 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:47:51.871277 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:47:51.871277 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 10 11:47:51.871277 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 10 11:47:51.871277 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 10 11:47:51.871277 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 10 11:47:51.873327 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 10 11:47:51.873327 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 10 11:47:51.873327 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 10 11:47:51.873327 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 10 11:47:51.873327 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:47:51.873327 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:47:51.873327 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:47:51.873327 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:47:51.873327 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:47:51.873327 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:47:51.873327 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:47:51.873327 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application with Lock( Oct 10 11:47:51.873327 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:47:51.873327 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:47:51.873327 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:47:51.873327 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:47:51.875103 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:47:51.875103 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:47:51.875103 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:47:51.875103 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:47:51.875103 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 10 11:47:51.875103 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 10 11:47:51.875103 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 10 11:47:51.875103 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 10 11:47:51.875103 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 10 11:47:51.875103 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 10 11:47:51.875103 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 10 11:47:51.875103 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Oct 10 11:47:51.880050 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 454/933] 10.222.0.46 () {56 vars in 1120 bytes} [Mon Oct 10 11:47:51 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:47:51.888281 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd31821-a38b-4730-8cd5-b32847ba5f18 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:52.040670 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:52.084574 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd31821-a38b-4730-8cd5-b32847ba5f18 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:52.156438 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d041d2a-dda5-4b40-8885-ddae78e87064 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:52.158046 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d041d2a-dda5-4b40-8885-ddae78e87064 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:52.158909 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d041d2a-dda5-4b40-8885-ddae78e87064 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:52.159784 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d041d2a-dda5-4b40-8885-ddae78e87064 None admin] PATH_INFO: `/v3/roles` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:52.161338 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d041d2a-dda5-4b40-8885-ddae78e87064 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:47:52.162193 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d041d2a-dda5-4b40-8885-ddae78e87064 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:52.165078 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d041d2a-dda5-4b40-8885-ddae78e87064 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:52.182459 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 480/934] 10.222.0.46 () {56 vars in 1115 bytes} [Mon Oct 10 11:47:51 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:47:52.190806 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d8c78f-eaf5-489a-99a2-b0d2c9759573 None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:52.300969 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:52.305320 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d8c78f-eaf5-489a-99a2-b0d2c9759573 None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:52.348008 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56db481b-a695-4588-8460-62f12f2556f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:52.348656 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56db481b-a695-4588-8460-62f12f2556f3 None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:52.348940 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56db481b-a695-4588-8460-62f12f2556f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:52.349209 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56db481b-a695-4588-8460-62f12f2556f3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:52.360522 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56db481b-a695-4588-8460-62f12f2556f3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:52.361995 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56db481b-a695-4588-8460-62f12f2556f3 None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:52.379419 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application [None req-56db481b-a695-4588-8460-62f12f2556f3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:47:52.379419 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:47:52.379419 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:47:52.379419 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:47:52.379419 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:47:52.379419 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:47:52.379419 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:47:52.379419 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:47:52.379419 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:47:52.379419 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:47:52.379419 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:47:52.379419 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:47:52.379419 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 10 11:47:52.379419 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 10 11:47:52.379419 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 10 11:47:52.379419 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 10 11:47:52.380457 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:47:52.380457 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:47:52.380457 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:47:52.380457 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:47:52.380457 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:47:52.380457 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:47:52.380457 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:47:52.380457 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application with Lock( Oct 10 11:47:52.380457 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:47:52.380457 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:47:52.380457 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:47:52.380457 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:47:52.380457 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:47:52.380457 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:47:52.380457 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:47:52.380457 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:47:52.380457 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 10 11:47:52.381398 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 10 11:47:52.381398 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:47:52.381398 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Oct 10 11:47:52.382121 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 455/935] 10.222.0.46 () {56 vars in 1108 bytes} [Mon Oct 10 11:47:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:47:52.390717 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d041d2a-dda5-4b40-8885-ddae78e87064 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:52.529052 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d041d2a-dda5-4b40-8885-ddae78e87064 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:52.573498 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ba3fa6-5222-4ae4-980c-def2a0a467ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:52.574030 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ba3fa6-5222-4ae4-980c-def2a0a467ac None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:52.574311 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ba3fa6-5222-4ae4-980c-def2a0a467ac None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:52.574586 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ba3fa6-5222-4ae4-980c-def2a0a467ac None admin] PATH_INFO: `/v3/domains` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:52.575276 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ba3fa6-5222-4ae4-980c-def2a0a467ac None admin] RBAC: Adding query filter params (name=Default) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:47:52.575579 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ba3fa6-5222-4ae4-980c-def2a0a467ac None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:52.576970 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ba3fa6-5222-4ae4-980c-def2a0a467ac None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:52.587770 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:52.592915 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 481/936] 10.222.0.46 () {56 vars in 1109 bytes} [Mon Oct 10 11:47:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:47:52.599608 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56db481b-a695-4588-8460-62f12f2556f3 None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:52.679814 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64727442-8aa8-43ec-a440-1fde5ca49bcc None None] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:52.680035 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64727442-8aa8-43ec-a440-1fde5ca49bcc None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:52.680122 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64727442-8aa8-43ec-a440-1fde5ca49bcc None None] PATH_INFO: `/` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:52.681825 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 482/937] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 10 11:47:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 10 11:47:52.689112 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83ddd9c-8a41-4582-b8f5-6281f220df3e None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:52.689406 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83ddd9c-8a41-4582-b8f5-6281f220df3e None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:52.689759 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83ddd9c-8a41-4582-b8f5-6281f220df3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:52.709599 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-a83ddd9c-8a41-4582-b8f5-6281f220df3e None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:47:52.710870 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-a83ddd9c-8a41-4582-b8f5-6281f220df3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:47:52.738843 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56db481b-a695-4588-8460-62f12f2556f3 None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:52.795098 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4739fac6-e2f6-48e0-81d7-e1a79478a325 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:52.796742 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4739fac6-e2f6-48e0-81d7-e1a79478a325 None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:52.797395 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4739fac6-e2f6-48e0-81d7-e1a79478a325 None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:52.797941 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4739fac6-e2f6-48e0-81d7-e1a79478a325 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:52.806772 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 483/938] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 10 11:47:52 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 10 11:47:52.807232 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4739fac6-e2f6-48e0-81d7-e1a79478a325 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:52.808953 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4c9654-7948-4713-842f-dff9af279196 None None] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:52.809256 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4c9654-7948-4713-842f-dff9af279196 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:52.809417 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4739fac6-e2f6-48e0-81d7-e1a79478a325 None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:52.809570 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4c9654-7948-4713-842f-dff9af279196 None None] PATH_INFO: `/` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:52.811643 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 484/939] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 10 11:47:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:47:52.820139 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2fce51-b7ca-41bf-bae3-29028a804627 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:52.820493 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2fce51-b7ca-41bf-bae3-29028a804627 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:52.820762 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2fce51-b7ca-41bf-bae3-29028a804627 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:52.831811 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application [None req-4739fac6-e2f6-48e0-81d7-e1a79478a325 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 10 11:47:52.831811 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:47:52.831811 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:47:52.831811 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:47:52.831811 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:47:52.831811 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:47:52.831811 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:47:52.831811 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:47:52.831811 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:47:52.831811 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:47:52.831811 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:47:52.831811 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:47:52.831811 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 10 11:47:52.831811 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 10 11:47:52.831811 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 10 11:47:52.831811 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 10 11:47:52.833575 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:47:52.833575 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:47:52.833575 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 10 11:47:52.833575 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 10 11:47:52.833575 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 10 11:47:52.833575 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 10 11:47:52.833575 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:47:52.833575 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:47:52.833575 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:47:52.833575 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:47:52.833575 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:47:52.833575 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application with Lock( Oct 10 11:47:52.833575 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:47:52.833575 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:47:52.833575 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:47:52.833575 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:47:52.833575 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:47:52.833575 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:47:52.835544 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:47:52.835544 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:47:52.835544 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 10 11:47:52.835544 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 10 11:47:52.835544 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 10 11:47:52.835544 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 10 11:47:52.835544 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 10 11:47:52.835544 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 10 11:47:52.835544 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 10 11:47:52.835544 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Oct 10 11:47:52.840263 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 456/940] 10.222.0.46 () {56 vars in 1117 bytes} [Mon Oct 10 11:47:52 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 10 11:47:52.841768 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-9d2fce51-b7ca-41bf-bae3-29028a804627 None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:47:52.844190 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-9d2fce51-b7ca-41bf-bae3-29028a804627 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:47:52.846799 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4739fac6-e2f6-48e0-81d7-e1a79478a325 None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:52.872037 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:52.950694 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 485/941] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 10 11:47:52 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 10 11:47:53.051321 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4739fac6-e2f6-48e0-81d7-e1a79478a325 None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:53.090448 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2fce51-b7ca-41bf-bae3-29028a804627 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:53.106728 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8dffffd-e697-4086-b046-346dc0183392 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:53.107788 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dffffd-e697-4086-b046-346dc0183392 None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:53.108286 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dffffd-e697-4086-b046-346dc0183392 None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:53.108984 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dffffd-e697-4086-b046-346dc0183392 None admin] PATH_INFO: `/v3/users` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:53.110234 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8dffffd-e697-4086-b046-346dc0183392 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:47:53.110858 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8dffffd-e697-4086-b046-346dc0183392 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:53.113581 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8dffffd-e697-4086-b046-346dc0183392 None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:53.145743 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 457/942] 10.222.0.46 () {56 vars in 1149 bytes} [Mon Oct 10 11:47:52 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 11:47:53.152738 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8dffffd-e697-4086-b046-346dc0183392 None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:53.188012 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:53.240718 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:53.260663 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2fce51-b7ca-41bf-bae3-29028a804627 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:53.285826 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8dffffd-e697-4086-b046-346dc0183392 None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:53.308302 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc69c5fb-0f18-4ac5-a69c-0b73c5db667a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:53.309310 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc69c5fb-0f18-4ac5-a69c-0b73c5db667a None admin] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:53.309750 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc69c5fb-0f18-4ac5-a69c-0b73c5db667a None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:53.310168 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc69c5fb-0f18-4ac5-a69c-0b73c5db667a None admin] PATH_INFO: `/v3/users` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:53.311085 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc69c5fb-0f18-4ac5-a69c-0b73c5db667a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:53.313015 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc69c5fb-0f18-4ac5-a69c-0b73c5db667a None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:53.316400 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: WARNING keystone.server.flask.common [None req-dc69c5fb-0f18-4ac5-a69c-0b73c5db667a None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Oct 10 11:47:53.338417 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b736aa1-89f3-4486-bda4-2f21e41cc692 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:53.343776 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b736aa1-89f3-4486-bda4-2f21e41cc692 None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:53.343776 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b736aa1-89f3-4486-bda4-2f21e41cc692 None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:53.343776 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b736aa1-89f3-4486-bda4-2f21e41cc692 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:53.353466 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b736aa1-89f3-4486-bda4-2f21e41cc692 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:53.355062 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b736aa1-89f3-4486-bda4-2f21e41cc692 None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:53.363549 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.sql.core [None req-dc69c5fb-0f18-4ac5-a69c-0b73c5db667a None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Oct 10 11:47:53.363549 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Oct 10 11:47:53.363549 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [parameters: {'user_id': 'a37028bdf3f44c14a60cb1079df1c1ee', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Oct 10 11:47:53.363549 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81677) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Oct 10 11:47:53.371646 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application [None req-3b736aa1-89f3-4486-bda4-2f21e41cc692 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:47:53.371646 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:47:53.371646 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:47:53.371646 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:47:53.371646 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:47:53.371646 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:47:53.371646 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:47:53.371646 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:47:53.371646 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:47:53.371646 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:47:53.371646 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:47:53.371646 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:47:53.371646 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 10 11:47:53.371646 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 10 11:47:53.371646 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 10 11:47:53.371646 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 10 11:47:53.372988 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:47:53.372988 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:47:53.372988 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:47:53.372988 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:47:53.372988 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:47:53.372988 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:47:53.372988 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:47:53.372988 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application with Lock( Oct 10 11:47:53.372988 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:47:53.372988 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:47:53.372988 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:47:53.372988 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:47:53.372988 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:47:53.372988 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:47:53.372988 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:47:53.372988 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:47:53.372988 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 10 11:47:53.381046 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 10 11:47:53.381046 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:47:53.381046 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Oct 10 11:47:53.381247 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application [None req-dc69c5fb-0f18-4ac5-a69c-0b73c5db667a None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Oct 10 11:47:53.381247 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:47:53.381247 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 10 11:47:53.381247 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 10 11:47:53.381247 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 10 11:47:53.381247 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 10 11:47:53.381247 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 10 11:47:53.381247 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application result = self._query(query) Oct 10 11:47:53.381247 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 10 11:47:53.381247 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application conn.query(q) Oct 10 11:47:53.381247 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 10 11:47:53.381247 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 10 11:47:53.381247 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 10 11:47:53.381247 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application result.read() Oct 10 11:47:53.381247 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 10 11:47:53.382708 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 10 11:47:53.382708 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 10 11:47:53.382708 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 10 11:47:53.382708 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 10 11:47:53.382708 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 10 11:47:53.382708 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 10 11:47:53.382708 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 10 11:47:53.382708 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Oct 10 11:47:53.382708 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Oct 10 11:47:53.382708 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Oct 10 11:47:53.382708 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Oct 10 11:47:53.382708 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:47:53.382708 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Oct 10 11:47:53.382708 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return method(*args, **kwargs) Oct 10 11:47:53.382708 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Oct 10 11:47:53.382708 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Oct 10 11:47:53.382708 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 10 11:47:53.382708 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application next(self.gen) Oct 10 11:47:53.384212 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Oct 10 11:47:53.384212 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application yield resource Oct 10 11:47:53.384212 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 10 11:47:53.384212 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application next(self.gen) Oct 10 11:47:53.384212 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Oct 10 11:47:53.384212 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application self.session.rollback() Oct 10 11:47:53.384212 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 10 11:47:53.384212 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application self.force_reraise() Oct 10 11:47:53.384212 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 10 11:47:53.384212 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise self.value Oct 10 11:47:53.384212 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Oct 10 11:47:53.384212 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Oct 10 11:47:53.384212 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Oct 10 11:47:53.384212 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application session.commit() Oct 10 11:47:53.384212 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Oct 10 11:47:53.384212 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Oct 10 11:47:53.384212 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Oct 10 11:47:53.387175 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application self._prepare_impl() Oct 10 11:47:53.387175 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Oct 10 11:47:53.387175 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application self.session.flush() Oct 10 11:47:53.387175 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Oct 10 11:47:53.387175 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application self._flush(objects) Oct 10 11:47:53.387175 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Oct 10 11:47:53.387175 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Oct 10 11:47:53.387175 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Oct 10 11:47:53.387175 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application compat.raise_( Oct 10 11:47:53.387175 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 10 11:47:53.387175 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception Oct 10 11:47:53.387175 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Oct 10 11:47:53.387175 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application flush_context.execute() Oct 10 11:47:53.387175 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Oct 10 11:47:53.387175 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application rec.execute(self) Oct 10 11:47:53.387175 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Oct 10 11:47:53.387175 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Oct 10 11:47:53.390377 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Oct 10 11:47:53.390377 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application _emit_insert_statements( Oct 10 11:47:53.390377 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Oct 10 11:47:53.390377 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application result = connection._execute_20( Oct 10 11:47:53.390377 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Oct 10 11:47:53.390377 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Oct 10 11:47:53.390377 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Oct 10 11:47:53.390377 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Oct 10 11:47:53.390377 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Oct 10 11:47:53.390377 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application ret = self._execute_context( Oct 10 11:47:53.390377 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Oct 10 11:47:53.390377 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Oct 10 11:47:53.390377 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Oct 10 11:47:53.390377 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Oct 10 11:47:53.390377 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 10 11:47:53.390377 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception Oct 10 11:47:53.393044 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 10 11:47:53.393044 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 10 11:47:53.393044 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 10 11:47:53.393044 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 10 11:47:53.393044 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 10 11:47:53.393044 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application result = self._query(query) Oct 10 11:47:53.393044 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 10 11:47:53.393044 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application conn.query(q) Oct 10 11:47:53.393044 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 10 11:47:53.393044 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 10 11:47:53.393044 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 10 11:47:53.393044 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application result.read() Oct 10 11:47:53.393044 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 10 11:47:53.393044 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 10 11:47:53.393044 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 10 11:47:53.393044 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 10 11:47:53.395103 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 10 11:47:53.395103 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 10 11:47:53.395103 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 10 11:47:53.395103 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 10 11:47:53.395103 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Oct 10 11:47:53.395103 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Oct 10 11:47:53.395103 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application [parameters: {'user_id': 'a37028bdf3f44c14a60cb1079df1c1ee', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Oct 10 11:47:53.395103 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Oct 10 11:47:53.395103 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Oct 10 11:47:53.395103 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 10 11:47:53.395103 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Oct 10 11:47:53.395103 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:47:53.395103 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:47:53.395103 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:47:53.395103 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:47:53.396932 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:47:53.396932 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:47:53.396932 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:47:53.396932 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:47:53.396932 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:47:53.396932 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:47:53.396932 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:47:53.396932 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Oct 10 11:47:53.396932 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Oct 10 11:47:53.396932 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:47:53.396932 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:47:53.396932 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 10 11:47:53.396932 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 10 11:47:53.396932 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 10 11:47:53.396932 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 10 11:47:53.396932 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Oct 10 11:47:53.396932 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Oct 10 11:47:53.399699 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Oct 10 11:47:53.399699 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Oct 10 11:47:53.399699 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Oct 10 11:47:53.399699 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Oct 10 11:47:53.399699 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Oct 10 11:47:53.399699 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Oct 10 11:47:53.400594 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 458/943] 10.222.0.46 () {56 vars in 1108 bytes} [Mon Oct 10 11:47:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 10 11:47:53.400594 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b736aa1-89f3-4486-bda4-2f21e41cc692 None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:53.400882 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 486/944] 10.222.0.46 () {60 vars in 1131 bytes} [Mon Oct 10 11:47:53 2022] POST /identity/v3/users => generated 160 bytes in 287 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Oct 10 11:47:53.400882 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc69c5fb-0f18-4ac5-a69c-0b73c5db667a None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:53.538495 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc69c5fb-0f18-4ac5-a69c-0b73c5db667a None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:53.541660 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b736aa1-89f3-4486-bda4-2f21e41cc692 None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:53.567494 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:53.582610 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474121df-ae54-4041-a414-242e3b3820d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:53.584056 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474121df-ae54-4041-a414-242e3b3820d2 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:53.584210 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:53.584367 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474121df-ae54-4041-a414-242e3b3820d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:53.584367 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474121df-ae54-4041-a414-242e3b3820d2 None admin] PATH_INFO: `/v3/domains` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:53.584653 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474121df-ae54-4041-a414-242e3b3820d2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:47:53.584906 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474121df-ae54-4041-a414-242e3b3820d2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:53.586435 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474121df-ae54-4041-a414-242e3b3820d2 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:53.612648 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 487/945] 10.222.0.46 () {56 vars in 1109 bytes} [Mon Oct 10 11:47:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 11:47:53.612648 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474121df-ae54-4041-a414-242e3b3820d2 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:53.615267 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab817105-05e8-412a-a958-f140b6286fad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:53.616296 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab817105-05e8-412a-a958-f140b6286fad None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:53.616763 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab817105-05e8-412a-a958-f140b6286fad None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:53.617114 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab817105-05e8-412a-a958-f140b6286fad None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:53.617706 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab817105-05e8-412a-a958-f140b6286fad None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:53.619794 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab817105-05e8-412a-a958-f140b6286fad None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:53.636441 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application [None req-ab817105-05e8-412a-a958-f140b6286fad None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 10 11:47:53.636441 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:47:53.636441 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 10 11:47:53.636441 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 10 11:47:53.636441 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 10 11:47:53.636441 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 10 11:47:53.636441 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 10 11:47:53.636441 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 10 11:47:53.636441 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 10 11:47:53.636441 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Oct 10 11:47:53.636441 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 10 11:47:53.636441 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Oct 10 11:47:53.636441 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:47:53.636441 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:47:53.636441 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:47:53.636441 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:47:53.645638 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:47:53.645638 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:47:53.645638 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:47:53.645638 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:47:53.645638 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:47:53.645638 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:47:53.645638 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:47:53.645638 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 10 11:47:53.645638 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 10 11:47:53.645638 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 10 11:47:53.645638 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 10 11:47:53.645638 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:47:53.645638 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:47:53.645638 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:47:53.645638 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:47:53.645638 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:47:53.645638 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:47:53.646881 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:47:53.646881 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application with Lock( Oct 10 11:47:53.646881 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:47:53.646881 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:47:53.646881 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:47:53.646881 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:47:53.646881 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:47:53.646881 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:47:53.646881 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:47:53.646881 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:47:53.646881 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 10 11:47:53.646881 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 10 11:47:53.646881 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 10 11:47:53.646881 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Oct 10 11:47:53.646881 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 459/946] 10.222.0.46 () {56 vars in 1132 bytes} [Mon Oct 10 11:47:53 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:47:53.648521 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab817105-05e8-412a-a958-f140b6286fad None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:53.762440 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474121df-ae54-4041-a414-242e3b3820d2 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:53.799785 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab817105-05e8-412a-a958-f140b6286fad None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:53.802474 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1d549f-7f20-4529-9c8b-8c670b20ecc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:53.803245 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1d549f-7f20-4529-9c8b-8c670b20ecc3 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:53.803528 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1d549f-7f20-4529-9c8b-8c670b20ecc3 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:53.803810 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1d549f-7f20-4529-9c8b-8c670b20ecc3 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:53.812467 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1d549f-7f20-4529-9c8b-8c670b20ecc3 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:53.813917 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1d549f-7f20-4529-9c8b-8c670b20ecc3 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:53.827807 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application [None req-cf1d549f-7f20-4529-9c8b-8c670b20ecc3 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Oct 10 11:47:53.827807 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:47:53.827807 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:47:53.827807 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:47:53.827807 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:47:53.827807 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:47:53.827807 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:47:53.827807 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:47:53.827807 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:47:53.827807 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:47:53.827807 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:47:53.827807 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:47:53.827807 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 10 11:47:53.827807 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 10 11:47:53.827807 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 10 11:47:53.827807 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 10 11:47:53.829109 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:47:53.829109 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:47:53.829109 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 10 11:47:53.829109 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 10 11:47:53.829109 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 10 11:47:53.829109 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 10 11:47:53.829109 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:47:53.829109 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:47:53.829109 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:47:53.829109 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:47:53.829109 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:47:53.829109 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application with Lock( Oct 10 11:47:53.829109 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:47:53.829109 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:47:53.829109 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:47:53.829109 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:47:53.829109 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:47:53.829109 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:47:53.830037 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:47:53.830037 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:47:53.830037 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 10 11:47:53.830037 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 10 11:47:53.830037 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 10 11:47:53.830037 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 10 11:47:53.830037 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 10 11:47:53.830037 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 10 11:47:53.830037 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Oct 10 11:47:53.830037 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Oct 10 11:47:53.830037 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 488/947] 10.222.0.46 () {56 vars in 1099 bytes} [Mon Oct 10 11:47:53 2022] GET /identity/v3/users/cinder => generated 84 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:47:53.830640 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1d549f-7f20-4529-9c8b-8c670b20ecc3 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:53.850161 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104d670b-66d2-4000-9b8f-7eddbfb9560a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:53.850866 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104d670b-66d2-4000-9b8f-7eddbfb9560a None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:53.851151 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104d670b-66d2-4000-9b8f-7eddbfb9560a None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:53.851431 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104d670b-66d2-4000-9b8f-7eddbfb9560a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:53.859431 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104d670b-66d2-4000-9b8f-7eddbfb9560a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:53.861011 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104d670b-66d2-4000-9b8f-7eddbfb9560a None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:53.874203 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application [None req-104d670b-66d2-4000-9b8f-7eddbfb9560a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 10 11:47:53.874203 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:47:53.874203 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:47:53.874203 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:47:53.874203 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:47:53.874203 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:47:53.874203 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:47:53.874203 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:47:53.874203 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:47:53.874203 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:47:53.874203 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:47:53.874203 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:47:53.874203 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 10 11:47:53.874203 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 10 11:47:53.874203 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 10 11:47:53.875891 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 10 11:47:53.875891 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:47:53.875891 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:47:53.875891 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:47:53.875891 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:47:53.875891 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:47:53.875891 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:47:53.875891 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:47:53.875891 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application with Lock( Oct 10 11:47:53.875891 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:47:53.875891 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:47:53.875891 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:47:53.875891 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:47:53.875891 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:47:53.875891 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:47:53.875891 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:47:53.875891 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:47:53.878217 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 10 11:47:53.878217 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 10 11:47:53.878217 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 10 11:47:53.878217 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 10 11:47:53.878217 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 10 11:47:53.878217 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 10 11:47:53.878217 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 10 11:47:53.878217 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Oct 10 11:47:53.878217 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 460/948] 10.222.0.46 () {56 vars in 1111 bytes} [Mon Oct 10 11:47:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:47:53.878217 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104d670b-66d2-4000-9b8f-7eddbfb9560a None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:53.906013 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:53.941165 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:53.996577 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1d549f-7f20-4529-9c8b-8c670b20ecc3 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:54.043939 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104d670b-66d2-4000-9b8f-7eddbfb9560a None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:54.054185 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1803ef6a-560b-4794-b7b2-d2bcb62a5a02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:54.055254 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1803ef6a-560b-4794-b7b2-d2bcb62a5a02 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:54.055710 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1803ef6a-560b-4794-b7b2-d2bcb62a5a02 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:54.056157 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1803ef6a-560b-4794-b7b2-d2bcb62a5a02 None admin] PATH_INFO: `/v3/services` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:54.057238 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1803ef6a-560b-4794-b7b2-d2bcb62a5a02 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:47:54.057690 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1803ef6a-560b-4794-b7b2-d2bcb62a5a02 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:54.060147 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1803ef6a-560b-4794-b7b2-d2bcb62a5a02 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:54.069047 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 489/949] 10.222.0.46 () {56 vars in 1126 bytes} [Mon Oct 10 11:47:53 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:47:54.070399 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1803ef6a-560b-4794-b7b2-d2bcb62a5a02 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:54.103585 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d5a679-679f-4149-9557-2bfb91aaa2c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:54.103585 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d5a679-679f-4149-9557-2bfb91aaa2c1 None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:54.103585 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d5a679-679f-4149-9557-2bfb91aaa2c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:54.103585 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d5a679-679f-4149-9557-2bfb91aaa2c1 None admin] PATH_INFO: `/v3/users` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:54.105033 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d5a679-679f-4149-9557-2bfb91aaa2c1 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=81676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:47:54.105033 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d5a679-679f-4149-9557-2bfb91aaa2c1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:54.105033 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d5a679-679f-4149-9557-2bfb91aaa2c1 None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:54.154650 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 461/950] 10.222.0.46 () {56 vars in 1101 bytes} [Mon Oct 10 11:47:53 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:47:54.157100 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979d3fe1-a29f-4f98-b9a7-d5d50c548c76 None None] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:54.157648 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979d3fe1-a29f-4f98-b9a7-d5d50c548c76 None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:54.158170 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979d3fe1-a29f-4f98-b9a7-d5d50c548c76 None None] PATH_INFO: `/` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:54.165579 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 462/951] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 10 11:47:54 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:47:54.165579 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979d3fe1-a29f-4f98-b9a7-d5d50c548c76 None None] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:54.243157 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1803ef6a-560b-4794-b7b2-d2bcb62a5a02 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:54.259954 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:54.310362 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd15470-4a95-4ebf-b400-82580f255860 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:54.311121 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd15470-4a95-4ebf-b400-82580f255860 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:54.311412 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd15470-4a95-4ebf-b400-82580f255860 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:54.311759 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd15470-4a95-4ebf-b400-82580f255860 None admin] PATH_INFO: `/v3/projects` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:54.312935 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd15470-4a95-4ebf-b400-82580f255860 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:47:54.313298 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd15470-4a95-4ebf-b400-82580f255860 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:54.316020 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd15470-4a95-4ebf-b400-82580f255860 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:54.333013 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 490/952] 10.222.0.46 () {56 vars in 1148 bytes} [Mon Oct 10 11:47:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:47:54.335840 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa89e6a-c2f6-4bc2-bbcc-693b2ef416fd None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:54.336315 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa89e6a-c2f6-4bc2-bbcc-693b2ef416fd None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:54.336800 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa89e6a-c2f6-4bc2-bbcc-693b2ef416fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:54.371948 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-efa89e6a-c2f6-4bc2-bbcc-693b2ef416fd None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:47:54.373393 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-efa89e6a-c2f6-4bc2-bbcc-693b2ef416fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:47:54.374102 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979d3fe1-a29f-4f98-b9a7-d5d50c548c76 None None] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:54.376552 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:54.428508 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2173b0a-46d6-404a-86eb-e463a6a09393 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:54.429379 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2173b0a-46d6-404a-86eb-e463a6a09393 None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:54.429666 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2173b0a-46d6-404a-86eb-e463a6a09393 None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:54.429921 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2173b0a-46d6-404a-86eb-e463a6a09393 None admin] PATH_INFO: `/v3/services` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:54.430583 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2173b0a-46d6-404a-86eb-e463a6a09393 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:47:54.430978 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2173b0a-46d6-404a-86eb-e463a6a09393 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:54.432308 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2173b0a-46d6-404a-86eb-e463a6a09393 None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:54.440629 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 463/953] 10.222.0.46 () {56 vars in 1126 bytes} [Mon Oct 10 11:47:54 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:47:54.440629 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2173b0a-46d6-404a-86eb-e463a6a09393 None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:54.509345 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 491/954] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 10 11:47:54 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 10 11:47:54.510229 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa89e6a-c2f6-4bc2-bbcc-693b2ef416fd None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:54.593191 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2173b0a-46d6-404a-86eb-e463a6a09393 None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:54.629299 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:54.635429 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa89e6a-c2f6-4bc2-bbcc-693b2ef416fd None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:54.660176 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b05a7e3-e1d7-4a26-98cc-9f876feba481 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:54.661281 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b05a7e3-e1d7-4a26-98cc-9f876feba481 None admin] REQUEST_METHOD: `PUT` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:54.661764 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b05a7e3-e1d7-4a26-98cc-9f876feba481 None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:54.662254 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b05a7e3-e1d7-4a26-98cc-9f876feba481 None admin] PATH_INFO: `/v3/projects/4ad3772f7b6a47c28b7ba37a9316a3c1/users/d6430593eecf45d8b891c3b55de4a944/roles/66c72919d8744554bc2e7c3dcc683e1f` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:54.666269 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b05a7e3-e1d7-4a26-98cc-9f876feba481 None admin] RBAC: Authorizing `identity:create_grant(project_id=4ad3772f7b6a47c28b7ba37a9316a3c1, user_id=d6430593eecf45d8b891c3b55de4a944, role_id=66c72919d8744554bc2e7c3dcc683e1f)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:54.669600 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b05a7e3-e1d7-4a26-98cc-9f876feba481 None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:54.677761 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9dbefb-5df9-4b11-bce2-c61258403611 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:54.678413 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9dbefb-5df9-4b11-bce2-c61258403611 None admin] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:54.678802 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9dbefb-5df9-4b11-bce2-c61258403611 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:54.679057 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9dbefb-5df9-4b11-bce2-c61258403611 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:54.679692 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9dbefb-5df9-4b11-bce2-c61258403611 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:54.681108 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9dbefb-5df9-4b11-bce2-c61258403611 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:54.781616 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 464/955] 10.222.0.46 () {58 vars in 1439 bytes} [Mon Oct 10 11:47:54 2022] PUT /identity/v3/projects/4ad3772f7b6a47c28b7ba37a9316a3c1/users/d6430593eecf45d8b891c3b55de4a944/roles/66c72919d8744554bc2e7c3dcc683e1f => generated 0 bytes in 341 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 11:47:54.818712 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 492/956] 10.222.0.46 () {60 vars in 1144 bytes} [Mon Oct 10 11:47:54 2022] POST /identity/v3/endpoints => generated 347 bytes in 301 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 11:47:54.843351 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b05a7e3-e1d7-4a26-98cc-9f876feba481 None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:54.900487 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:55.044343 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b05a7e3-e1d7-4a26-98cc-9f876feba481 None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:55.092964 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e48762-eb31-4345-970d-98728462ac1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:55.093613 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e48762-eb31-4345-970d-98728462ac1c None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:55.093889 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e48762-eb31-4345-970d-98728462ac1c None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:55.094162 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e48762-eb31-4345-970d-98728462ac1c None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:55.094787 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e48762-eb31-4345-970d-98728462ac1c None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:55.096191 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e48762-eb31-4345-970d-98728462ac1c None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:55.112766 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application [None req-06e48762-eb31-4345-970d-98728462ac1c None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 10 11:47:55.112766 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:47:55.112766 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 10 11:47:55.112766 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 10 11:47:55.112766 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 10 11:47:55.112766 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 10 11:47:55.112766 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 10 11:47:55.112766 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 10 11:47:55.112766 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 10 11:47:55.112766 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Oct 10 11:47:55.112766 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 10 11:47:55.112766 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Oct 10 11:47:55.112766 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:47:55.112766 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:47:55.112766 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:47:55.112766 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:47:55.112766 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:47:55.113963 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:47:55.113963 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:47:55.113963 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:47:55.113963 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:47:55.113963 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:47:55.113963 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:47:55.113963 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 10 11:47:55.113963 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 10 11:47:55.113963 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 10 11:47:55.113963 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 10 11:47:55.113963 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:47:55.113963 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:47:55.113963 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:47:55.113963 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:47:55.113963 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:47:55.113963 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:47:55.115950 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:47:55.115950 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application with Lock( Oct 10 11:47:55.115950 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:47:55.115950 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:47:55.115950 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:47:55.115950 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:47:55.115950 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:47:55.115950 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:47:55.115950 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:47:55.115950 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:47:55.115950 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 10 11:47:55.115950 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 10 11:47:55.115950 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 10 11:47:55.115950 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Oct 10 11:47:55.115950 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 465/957] 10.222.0.46 () {56 vars in 1126 bytes} [Mon Oct 10 11:47:54 2022] GET /identity/v3/services/object-store => generated 93 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:47:55.121712 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9dbefb-5df9-4b11-bce2-c61258403611 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:55.235355 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9dbefb-5df9-4b11-bce2-c61258403611 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:55.275574 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf61ef2-1f46-4633-9057-bee880d79c2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:55.276913 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf61ef2-1f46-4633-9057-bee880d79c2f None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:55.277331 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf61ef2-1f46-4633-9057-bee880d79c2f None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:55.277726 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf61ef2-1f46-4633-9057-bee880d79c2f None admin] PATH_INFO: `/v3/services` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:55.278548 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf61ef2-1f46-4633-9057-bee880d79c2f None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:47:55.279014 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf61ef2-1f46-4633-9057-bee880d79c2f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:55.280633 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf61ef2-1f46-4633-9057-bee880d79c2f None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:55.293575 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 493/958] 10.222.0.46 () {56 vars in 1122 bytes} [Mon Oct 10 11:47:55 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:47:55.293770 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e48762-eb31-4345-970d-98728462ac1c None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:55.418308 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:55.436801 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e48762-eb31-4345-970d-98728462ac1c None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:55.492868 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e28cd2-d7b1-41c6-9d44-6edf804ebeca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:55.494179 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e28cd2-d7b1-41c6-9d44-6edf804ebeca None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:55.498927 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e28cd2-d7b1-41c6-9d44-6edf804ebeca None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:55.498927 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e28cd2-d7b1-41c6-9d44-6edf804ebeca None admin] PATH_INFO: `/v3/services` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:55.498927 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e28cd2-d7b1-41c6-9d44-6edf804ebeca None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:47:55.498927 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e28cd2-d7b1-41c6-9d44-6edf804ebeca None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:55.500200 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e28cd2-d7b1-41c6-9d44-6edf804ebeca None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:55.514190 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 466/959] 10.222.0.46 () {56 vars in 1122 bytes} [Mon Oct 10 11:47:55 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:47:55.516331 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf61ef2-1f46-4633-9057-bee880d79c2f None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:55.551160 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:55.712998 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf61ef2-1f46-4633-9057-bee880d79c2f None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:55.780244 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7945636d-ff12-4828-8088-85d74a42ccb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:55.781240 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7945636d-ff12-4828-8088-85d74a42ccb7 None admin] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:55.781691 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7945636d-ff12-4828-8088-85d74a42ccb7 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:55.782171 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7945636d-ff12-4828-8088-85d74a42ccb7 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:55.783476 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7945636d-ff12-4828-8088-85d74a42ccb7 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:55.786107 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7945636d-ff12-4828-8088-85d74a42ccb7 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:55.894713 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 494/960] 10.222.0.46 () {60 vars in 1144 bytes} [Mon Oct 10 11:47:55 2022] POST /identity/v3/endpoints => generated 349 bytes in 373 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 11:47:57.283845 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bf8f9c-cd92-4ad4-bb39-a2f685549832 None None] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:57.284405 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bf8f9c-cd92-4ad4-bb39-a2f685549832 None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:57.284405 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bf8f9c-cd92-4ad4-bb39-a2f685549832 None None] PATH_INFO: `/` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:57.285964 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 467/961] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 10 11:47:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:47:57.291691 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979fb02b-2885-497c-b7b8-168738505108 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:57.292201 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979fb02b-2885-497c-b7b8-168738505108 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:57.292393 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979fb02b-2885-497c-b7b8-168738505108 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:57.311344 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-979fb02b-2885-497c-b7b8-168738505108 None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:47:57.312565 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-979fb02b-2885-497c-b7b8-168738505108 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:47:57.386810 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:57.394913 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 495/962] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 10 11:47:57 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 10 11:47:57.508642 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5181654b-c4a4-443d-8b27-8463bca54aba None None] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:57.508990 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5181654b-c4a4-443d-8b27-8463bca54aba None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:57.509216 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5181654b-c4a4-443d-8b27-8463bca54aba None None] PATH_INFO: `/` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:57.511464 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 468/963] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 10 11:47:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:47:57.516924 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b76705-334f-4369-b912-006990dfde6b None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:57.517213 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b76705-334f-4369-b912-006990dfde6b None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:57.517498 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b76705-334f-4369-b912-006990dfde6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:57.537700 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-d7b76705-334f-4369-b912-006990dfde6b None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:47:57.539003 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-d7b76705-334f-4369-b912-006990dfde6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:47:57.611135 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 496/964] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 10 11:47:57 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 10 11:47:57.666892 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5181654b-c4a4-443d-8b27-8463bca54aba None None] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:57.779303 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5181654b-c4a4-443d-8b27-8463bca54aba None None] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:57.815207 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74de257a-7eea-487c-8396-1819b0cd285d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:57.815777 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74de257a-7eea-487c-8396-1819b0cd285d None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:57.816062 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74de257a-7eea-487c-8396-1819b0cd285d None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:57.816340 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74de257a-7eea-487c-8396-1819b0cd285d None admin] PATH_INFO: `/v3/services/compute` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:57.816936 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74de257a-7eea-487c-8396-1819b0cd285d None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:57.818382 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74de257a-7eea-487c-8396-1819b0cd285d None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:57.828351 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application [None req-74de257a-7eea-487c-8396-1819b0cd285d None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Oct 10 11:47:57.828351 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:47:57.828351 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 10 11:47:57.828351 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 10 11:47:57.828351 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 10 11:47:57.828351 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 10 11:47:57.828351 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 10 11:47:57.828351 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 10 11:47:57.828351 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 10 11:47:57.828351 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Oct 10 11:47:57.828351 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 10 11:47:57.828351 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Oct 10 11:47:57.828351 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:47:57.828351 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:47:57.828351 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:47:57.828351 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:47:57.828351 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:47:57.829347 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:47:57.829347 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:47:57.829347 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:47:57.829347 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:47:57.829347 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:47:57.829347 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:47:57.829347 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 10 11:47:57.829347 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 10 11:47:57.829347 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 10 11:47:57.829347 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 10 11:47:57.829347 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:47:57.829347 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:47:57.829347 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:47:57.829347 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:47:57.829347 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:47:57.829347 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:47:57.830198 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:47:57.830198 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application with Lock( Oct 10 11:47:57.830198 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:47:57.830198 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:47:57.830198 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:47:57.830198 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:47:57.830198 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:47:57.830198 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:47:57.830198 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:47:57.830198 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:47:57.830198 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 10 11:47:57.830198 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 10 11:47:57.830198 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 10 11:47:57.830198 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Oct 10 11:47:57.834447 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 469/965] 10.222.0.46 () {56 vars in 1111 bytes} [Mon Oct 10 11:47:57 2022] GET /identity/v3/services/compute => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:47:57.835836 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b76705-334f-4369-b912-006990dfde6b None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:57.890756 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74de257a-7eea-487c-8396-1819b0cd285d None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:57.909545 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:57.955640 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b76705-334f-4369-b912-006990dfde6b None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:57.991510 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:58.008438 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3542b548-f5c9-4e38-8b65-9047adbd071f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:58.010032 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3542b548-f5c9-4e38-8b65-9047adbd071f None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:58.010835 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3542b548-f5c9-4e38-8b65-9047adbd071f None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:58.011189 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3542b548-f5c9-4e38-8b65-9047adbd071f None admin] PATH_INFO: `/v3/services` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:58.012448 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3542b548-f5c9-4e38-8b65-9047adbd071f None admin] RBAC: Adding query filter params (name=compute) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:47:58.012921 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3542b548-f5c9-4e38-8b65-9047adbd071f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:58.015280 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3542b548-f5c9-4e38-8b65-9047adbd071f None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:58.024717 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 497/966] 10.222.0.46 () {56 vars in 1112 bytes} [Mon Oct 10 11:47:57 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 11:47:58.033553 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3542b548-f5c9-4e38-8b65-9047adbd071f None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:58.039792 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74de257a-7eea-487c-8396-1819b0cd285d None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:58.082368 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac61c99c-a408-43d2-8551-a416fb3a4f45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:58.082947 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac61c99c-a408-43d2-8551-a416fb3a4f45 None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:58.083232 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac61c99c-a408-43d2-8551-a416fb3a4f45 None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:58.083511 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac61c99c-a408-43d2-8551-a416fb3a4f45 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:58.096910 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac61c99c-a408-43d2-8551-a416fb3a4f45 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 10 11:47:58.102061 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application [None req-ac61c99c-a408-43d2-8551-a416fb3a4f45 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 10 11:47:58.102061 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:47:58.102061 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:47:58.102061 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:47:58.102061 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:47:58.102061 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:47:58.102061 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:47:58.102061 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:47:58.102061 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:47:58.102061 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:47:58.102061 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:47:58.102061 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:47:58.102061 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 10 11:47:58.102061 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 10 11:47:58.102061 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 10 11:47:58.102061 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 10 11:47:58.105251 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 10 11:47:58.105251 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 10 11:47:58.105251 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 10 11:47:58.105251 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 10 11:47:58.105251 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:47:58.105251 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:47:58.105251 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:47:58.105251 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:47:58.105251 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:47:58.105251 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:47:58.105251 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:47:58.105251 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application with Lock( Oct 10 11:47:58.105251 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:47:58.105251 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:47:58.105251 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:47:58.105251 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:47:58.107227 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:47:58.107227 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:47:58.107227 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:47:58.107227 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:47:58.107227 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 10 11:47:58.107227 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 10 11:47:58.107227 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 10 11:47:58.107227 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 10 11:47:58.107227 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 10 11:47:58.107227 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 10 11:47:58.107227 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 10 11:47:58.107227 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Oct 10 11:47:58.107227 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 470/967] 10.222.0.46 () {56 vars in 1120 bytes} [Mon Oct 10 11:47:57 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 10 11:47:58.110151 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac61c99c-a408-43d2-8551-a416fb3a4f45 None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:58.216786 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:58.252731 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3542b548-f5c9-4e38-8b65-9047adbd071f None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:58.262392 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac61c99c-a408-43d2-8551-a416fb3a4f45 None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:58.305080 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034a4931-03cd-4382-97d1-dd8e4443cf77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:58.306054 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034a4931-03cd-4382-97d1-dd8e4443cf77 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:58.306526 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034a4931-03cd-4382-97d1-dd8e4443cf77 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:58.307029 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034a4931-03cd-4382-97d1-dd8e4443cf77 None admin] PATH_INFO: `/v3/services` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:58.308116 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034a4931-03cd-4382-97d1-dd8e4443cf77 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:47:58.308566 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034a4931-03cd-4382-97d1-dd8e4443cf77 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:58.310135 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034a4931-03cd-4382-97d1-dd8e4443cf77 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:58.326731 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 498/968] 10.222.0.46 () {56 vars in 1112 bytes} [Mon Oct 10 11:47:58 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 11:47:58.348845 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad852539-34c9-45b9-a978-645943147c32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:58.349961 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad852539-34c9-45b9-a978-645943147c32 None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:58.350611 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad852539-34c9-45b9-a978-645943147c32 None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:58.351218 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad852539-34c9-45b9-a978-645943147c32 None admin] PATH_INFO: `/v3/roles` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:58.352835 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad852539-34c9-45b9-a978-645943147c32 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:47:58.353548 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad852539-34c9-45b9-a978-645943147c32 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:58.356975 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad852539-34c9-45b9-a978-645943147c32 None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:58.382511 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 471/969] 10.222.0.46 () {56 vars in 1115 bytes} [Mon Oct 10 11:47:58 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 11:47:58.384776 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034a4931-03cd-4382-97d1-dd8e4443cf77 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:58.505829 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:58.567353 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034a4931-03cd-4382-97d1-dd8e4443cf77 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:58.629898 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4a29ad-77a8-4029-8070-3f6607042285 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:58.631347 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4a29ad-77a8-4029-8070-3f6607042285 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:58.631990 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4a29ad-77a8-4029-8070-3f6607042285 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:58.632596 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4a29ad-77a8-4029-8070-3f6607042285 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:58.654932 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4a29ad-77a8-4029-8070-3f6607042285 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:58.657816 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4a29ad-77a8-4029-8070-3f6607042285 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:58.685671 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application [None req-6c4a29ad-77a8-4029-8070-3f6607042285 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:47:58.685671 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:47:58.685671 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:47:58.685671 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:47:58.685671 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:47:58.685671 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:47:58.685671 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:47:58.685671 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:47:58.685671 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:47:58.685671 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:47:58.685671 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:47:58.685671 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:47:58.685671 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 10 11:47:58.685671 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 10 11:47:58.685671 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 10 11:47:58.685671 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 10 11:47:58.687909 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:47:58.687909 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:47:58.687909 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:47:58.687909 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:47:58.687909 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:47:58.687909 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:47:58.687909 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:47:58.687909 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application with Lock( Oct 10 11:47:58.687909 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:47:58.687909 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:47:58.687909 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:47:58.687909 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:47:58.687909 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:47:58.687909 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:47:58.687909 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:47:58.687909 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:47:58.687909 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 10 11:47:58.689658 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 10 11:47:58.689658 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:47:58.689658 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Oct 10 11:47:58.689658 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 499/970] 10.222.0.46 () {56 vars in 1108 bytes} [Mon Oct 10 11:47:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 305 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:47:58.695089 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad852539-34c9-45b9-a978-645943147c32 None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:58.701299 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49048ff-f664-4dbd-9eb6-d1fa150cc5ec None None] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:58.702488 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49048ff-f664-4dbd-9eb6-d1fa150cc5ec None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:58.702488 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49048ff-f664-4dbd-9eb6-d1fa150cc5ec None None] PATH_INFO: `/` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:58.703386 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 500/971] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 10 11:47:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 10 11:47:58.708928 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa0084b-39a5-4b43-a9cd-07bbe78f1cea None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:58.709245 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa0084b-39a5-4b43-a9cd-07bbe78f1cea None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:58.709463 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa0084b-39a5-4b43-a9cd-07bbe78f1cea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:58.730748 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-bfa0084b-39a5-4b43-a9cd-07bbe78f1cea None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:47:58.732012 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-bfa0084b-39a5-4b43-a9cd-07bbe78f1cea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:47:58.817173 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 501/972] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 10 11:47:58 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 10 11:47:58.867989 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad852539-34c9-45b9-a978-645943147c32 None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:58.913876 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680688f8-6095-4287-b702-f24af9bbeb65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:58.914724 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680688f8-6095-4287-b702-f24af9bbeb65 None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:58.915149 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680688f8-6095-4287-b702-f24af9bbeb65 None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:58.915568 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680688f8-6095-4287-b702-f24af9bbeb65 None admin] PATH_INFO: `/v3/domains` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:58.916412 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680688f8-6095-4287-b702-f24af9bbeb65 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:47:58.916911 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680688f8-6095-4287-b702-f24af9bbeb65 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:58.918571 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680688f8-6095-4287-b702-f24af9bbeb65 None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:58.922152 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:58.939915 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 472/973] 10.222.0.46 () {56 vars in 1109 bytes} [Mon Oct 10 11:47:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 11:47:58.941772 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa0084b-39a5-4b43-a9cd-07bbe78f1cea None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:59.019205 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:59.133421 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa0084b-39a5-4b43-a9cd-07bbe78f1cea None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:59.160597 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680688f8-6095-4287-b702-f24af9bbeb65 None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:59.205667 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267c75ad-d654-4937-bd6b-8e67c6a9951b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:59.207498 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267c75ad-d654-4937-bd6b-8e67c6a9951b None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:59.207989 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267c75ad-d654-4937-bd6b-8e67c6a9951b None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:59.208475 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267c75ad-d654-4937-bd6b-8e67c6a9951b None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:59.222656 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267c75ad-d654-4937-bd6b-8e67c6a9951b None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:59.225553 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267c75ad-d654-4937-bd6b-8e67c6a9951b None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:59.250059 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application [None req-267c75ad-d654-4937-bd6b-8e67c6a9951b None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 10 11:47:59.250059 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:47:59.250059 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:47:59.250059 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:47:59.250059 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:47:59.250059 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:47:59.250059 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:47:59.250059 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:47:59.250059 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:47:59.250059 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:47:59.250059 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:47:59.250059 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:47:59.250059 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 10 11:47:59.250059 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 10 11:47:59.250059 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 10 11:47:59.250059 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 10 11:47:59.251982 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:47:59.251982 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:47:59.251982 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 10 11:47:59.251982 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 10 11:47:59.251982 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 10 11:47:59.251982 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 10 11:47:59.251982 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:47:59.251982 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:47:59.251982 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:47:59.251982 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:47:59.251982 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:47:59.251982 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application with Lock( Oct 10 11:47:59.251982 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:47:59.251982 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:47:59.251982 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:47:59.251982 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:47:59.251982 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:47:59.251982 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:47:59.253870 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:47:59.253870 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:47:59.253870 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 10 11:47:59.253870 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 10 11:47:59.253870 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 10 11:47:59.253870 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 10 11:47:59.253870 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 10 11:47:59.253870 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 10 11:47:59.253870 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 10 11:47:59.253870 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Oct 10 11:47:59.253870 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 502/974] 10.222.0.46 () {56 vars in 1117 bytes} [Mon Oct 10 11:47:58 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 311 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:47:59.258031 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267c75ad-d654-4937-bd6b-8e67c6a9951b None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:59.307032 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680688f8-6095-4287-b702-f24af9bbeb65 None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:59.366045 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc21ee5-1c42-4755-8693-f40ab1098987 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:59.367144 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc21ee5-1c42-4755-8693-f40ab1098987 None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:59.367501 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc21ee5-1c42-4755-8693-f40ab1098987 None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:59.367932 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc21ee5-1c42-4755-8693-f40ab1098987 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:59.368959 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc21ee5-1c42-4755-8693-f40ab1098987 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:59.371816 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc21ee5-1c42-4755-8693-f40ab1098987 None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:59.387769 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application [None req-fbc21ee5-1c42-4755-8693-f40ab1098987 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 10 11:47:59.387769 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:47:59.387769 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 10 11:47:59.387769 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 10 11:47:59.387769 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 10 11:47:59.387769 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 10 11:47:59.387769 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 10 11:47:59.387769 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 10 11:47:59.387769 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 10 11:47:59.387769 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Oct 10 11:47:59.387769 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 10 11:47:59.387769 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Oct 10 11:47:59.387769 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:47:59.387769 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:47:59.387769 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:47:59.387769 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:47:59.387769 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:47:59.389750 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:47:59.389750 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:47:59.389750 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:47:59.389750 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:47:59.389750 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:47:59.389750 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:47:59.389750 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 10 11:47:59.389750 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 10 11:47:59.389750 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 10 11:47:59.389750 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 10 11:47:59.389750 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:47:59.389750 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:47:59.389750 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:47:59.389750 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:47:59.389750 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:47:59.389750 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:47:59.391698 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:47:59.391698 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application with Lock( Oct 10 11:47:59.391698 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:47:59.391698 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:47:59.391698 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:47:59.391698 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:47:59.391698 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:47:59.391698 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:47:59.391698 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:47:59.391698 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:47:59.391698 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 10 11:47:59.391698 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 10 11:47:59.391698 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 10 11:47:59.391698 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Oct 10 11:47:59.391698 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 473/975] 10.222.0.46 () {56 vars in 1126 bytes} [Mon Oct 10 11:47:59 2022] GET /identity/v3/services/object-store => generated 93 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 10 11:47:59.396763 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc21ee5-1c42-4755-8693-f40ab1098987 None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:59.424217 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:59.430131 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267c75ad-d654-4937-bd6b-8e67c6a9951b None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:59.501016 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cae595-8e80-4635-bc1a-87745689c462 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:59.503997 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cae595-8e80-4635-bc1a-87745689c462 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:59.503997 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cae595-8e80-4635-bc1a-87745689c462 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:59.504276 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:59.504411 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cae595-8e80-4635-bc1a-87745689c462 None admin] PATH_INFO: `/v3/users` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:59.504411 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76cae595-8e80-4635-bc1a-87745689c462 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:47:59.504411 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76cae595-8e80-4635-bc1a-87745689c462 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:59.506216 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76cae595-8e80-4635-bc1a-87745689c462 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:59.539421 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 503/976] 10.222.0.46 () {56 vars in 1149 bytes} [Mon Oct 10 11:47:59 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 11:47:59.548295 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cae595-8e80-4635-bc1a-87745689c462 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:59.575932 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc21ee5-1c42-4755-8693-f40ab1098987 None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:59.637229 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366919c1-d042-45a4-a340-1eb8a86606b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:59.637851 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366919c1-d042-45a4-a340-1eb8a86606b2 None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:59.638131 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366919c1-d042-45a4-a340-1eb8a86606b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:59.638410 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366919c1-d042-45a4-a340-1eb8a86606b2 None admin] PATH_INFO: `/v3/services` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:59.639090 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366919c1-d042-45a4-a340-1eb8a86606b2 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:47:59.639399 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366919c1-d042-45a4-a340-1eb8a86606b2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:59.640794 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366919c1-d042-45a4-a340-1eb8a86606b2 None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:59.650771 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 474/977] 10.222.0.46 () {56 vars in 1122 bytes} [Mon Oct 10 11:47:59 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 11:47:59.653341 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366919c1-d042-45a4-a340-1eb8a86606b2 None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:59.700590 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cae595-8e80-4635-bc1a-87745689c462 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:59.765103 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82513544-a3a3-472e-b7d2-642d95ceaf87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:59.765993 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82513544-a3a3-472e-b7d2-642d95ceaf87 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:59.766439 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82513544-a3a3-472e-b7d2-642d95ceaf87 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:59.766932 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82513544-a3a3-472e-b7d2-642d95ceaf87 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:59.813290 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366919c1-d042-45a4-a340-1eb8a86606b2 None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:47:59.815624 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:59.845071 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:47:59.861859 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f0291f-3b3d-42c3-b349-fd70f555f758 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:47:59.862834 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f0291f-3b3d-42c3-b349-fd70f555f758 None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:47:59.863289 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f0291f-3b3d-42c3-b349-fd70f555f758 None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:47:59.863923 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f0291f-3b3d-42c3-b349-fd70f555f758 None admin] PATH_INFO: `/v3/services` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:47:59.864944 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f0291f-3b3d-42c3-b349-fd70f555f758 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:47:59.865444 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f0291f-3b3d-42c3-b349-fd70f555f758 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:59.867834 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f0291f-3b3d-42c3-b349-fd70f555f758 None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:47:59.879116 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 475/978] 10.222.0.46 () {56 vars in 1122 bytes} [Mon Oct 10 11:47:59 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 11:47:59.884081 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82513544-a3a3-472e-b7d2-642d95ceaf87 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:47:59.886795 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f0291f-3b3d-42c3-b349-fd70f555f758 None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:47:59.887301 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82513544-a3a3-472e-b7d2-642d95ceaf87 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:48:00.024583 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 504/979] 10.222.0.46 () {58 vars in 1282 bytes} [Mon Oct 10 11:47:59 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 477 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Oct 10 11:48:00.030927 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82513544-a3a3-472e-b7d2-642d95ceaf87 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:48:00.034883 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f0291f-3b3d-42c3-b349-fd70f555f758 None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:48:00.093663 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c291d3-5ea6-4c7f-a461-99e0ed95110a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:48:00.094314 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c291d3-5ea6-4c7f-a461-99e0ed95110a None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:48:00.094598 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c291d3-5ea6-4c7f-a461-99e0ed95110a None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:48:00.094948 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c291d3-5ea6-4c7f-a461-99e0ed95110a None admin] PATH_INFO: `/v3/endpoints` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:48:00.095844 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c291d3-5ea6-4c7f-a461-99e0ed95110a None admin] RBAC: Adding query filter params (interface=admin, service_id=bebea60cce1243768657f317c2f37e04, region_id=RegionOne) {{(pid=81676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:48:00.096360 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c291d3-5ea6-4c7f-a461-99e0ed95110a None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:48:00.098724 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c291d3-5ea6-4c7f-a461-99e0ed95110a None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:48:00.111144 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 476/980] 10.222.0.46 () {56 vars in 1249 bytes} [Mon Oct 10 11:47:59 2022] GET /identity/v3/endpoints?service_id=bebea60cce1243768657f317c2f37e04&interface=admin®ion_id=RegionOne => generated 193 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:48:00.117833 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c291d3-5ea6-4c7f-a461-99e0ed95110a None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:48:00.133769 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:48:00.207144 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:48:00.221471 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82513544-a3a3-472e-b7d2-642d95ceaf87 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:48:00.288131 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c291d3-5ea6-4c7f-a461-99e0ed95110a None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:48:00.294702 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a85f5b-2d29-446c-8e70-9b78770d0dce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:48:00.296008 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a85f5b-2d29-446c-8e70-9b78770d0dce None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:48:00.296684 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a85f5b-2d29-446c-8e70-9b78770d0dce None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:48:00.297341 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a85f5b-2d29-446c-8e70-9b78770d0dce None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:48:00.319667 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a85f5b-2d29-446c-8e70-9b78770d0dce None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:48:00.323814 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a85f5b-2d29-446c-8e70-9b78770d0dce None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:48:00.346268 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46151700-b15b-4995-a691-00efec1ccd3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:48:00.347298 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46151700-b15b-4995-a691-00efec1ccd3f None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:48:00.347774 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46151700-b15b-4995-a691-00efec1ccd3f None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:48:00.348245 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46151700-b15b-4995-a691-00efec1ccd3f None admin] PATH_INFO: `/v3/services` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:48:00.349336 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46151700-b15b-4995-a691-00efec1ccd3f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:48:00.351979 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46151700-b15b-4995-a691-00efec1ccd3f None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:48:00.356539 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application [None req-c4a85f5b-2d29-446c-8e70-9b78770d0dce None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:48:00.356539 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:48:00.356539 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:48:00.356539 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:48:00.356539 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:48:00.356539 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:48:00.356539 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:48:00.356539 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:48:00.356539 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:48:00.356539 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:48:00.356539 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:48:00.356539 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:48:00.356539 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 10 11:48:00.356539 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 10 11:48:00.356539 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 10 11:48:00.356539 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 10 11:48:00.358153 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:48:00.358153 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:48:00.358153 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:48:00.358153 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:48:00.358153 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:48:00.358153 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:48:00.358153 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:48:00.358153 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application with Lock( Oct 10 11:48:00.358153 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:48:00.358153 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:48:00.358153 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:48:00.358153 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:48:00.358153 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:48:00.358153 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:48:00.358153 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:48:00.358153 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:48:00.358153 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 10 11:48:00.359182 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 10 11:48:00.359182 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:48:00.359182 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Oct 10 11:48:00.360227 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 505/981] 10.222.0.46 () {56 vars in 1108 bytes} [Mon Oct 10 11:48:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 330 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 10 11:48:00.366245 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a85f5b-2d29-446c-8e70-9b78770d0dce None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:48:00.374799 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 477/982] 10.222.0.46 () {56 vars in 1087 bytes} [Mon Oct 10 11:48:00 2022] GET /identity/v3/services => generated 1497 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 10 11:48:00.572303 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a85f5b-2d29-446c-8e70-9b78770d0dce None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:48:00.590375 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:48:00.623968 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f7e83a-3970-4304-b258-8dd4a9749405 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:48:00.624597 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f7e83a-3970-4304-b258-8dd4a9749405 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:48:00.624876 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f7e83a-3970-4304-b258-8dd4a9749405 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:48:00.625156 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f7e83a-3970-4304-b258-8dd4a9749405 None admin] PATH_INFO: `/v3/domains` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:48:00.625810 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f7e83a-3970-4304-b258-8dd4a9749405 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:48:00.626114 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f7e83a-3970-4304-b258-8dd4a9749405 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:48:00.627545 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f7e83a-3970-4304-b258-8dd4a9749405 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:48:00.643352 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 506/983] 10.222.0.46 () {56 vars in 1109 bytes} [Mon Oct 10 11:48:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 11:48:00.652597 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46151700-b15b-4995-a691-00efec1ccd3f None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:48:00.770821 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46151700-b15b-4995-a691-00efec1ccd3f None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:48:00.790756 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:48:00.812447 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7baa28db-75ed-4611-b38b-21f1348601df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:48:00.814058 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ff1dd5-0b99-4fc4-b258-c072089c754e None None] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:48:00.814338 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ff1dd5-0b99-4fc4-b258-c072089c754e None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:48:00.814611 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ff1dd5-0b99-4fc4-b258-c072089c754e None None] PATH_INFO: `/` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:48:00.815998 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 507/984] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 10 11:48:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 10 11:48:00.816634 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7baa28db-75ed-4611-b38b-21f1348601df None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:48:00.816984 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7baa28db-75ed-4611-b38b-21f1348601df None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:48:00.817228 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7baa28db-75ed-4611-b38b-21f1348601df None admin] PATH_INFO: `/v3/projects/service` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:48:00.823426 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764305c1-27ba-49d9-88be-52ccb786b56a None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:48:00.823928 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764305c1-27ba-49d9-88be-52ccb786b56a None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:48:00.824283 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764305c1-27ba-49d9-88be-52ccb786b56a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:48:00.827295 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7baa28db-75ed-4611-b38b-21f1348601df None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:48:00.828849 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7baa28db-75ed-4611-b38b-21f1348601df None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:48:00.842578 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application [None req-7baa28db-75ed-4611-b38b-21f1348601df None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 10 11:48:00.842578 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:48:00.842578 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:48:00.842578 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:48:00.842578 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:48:00.842578 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:48:00.842578 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:48:00.842578 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:48:00.842578 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:48:00.842578 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:48:00.842578 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:48:00.842578 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:48:00.842578 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 10 11:48:00.842578 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 10 11:48:00.842578 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 10 11:48:00.844358 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 10 11:48:00.844358 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:48:00.844358 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:48:00.844358 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:48:00.844358 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:48:00.844358 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:48:00.844358 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:48:00.844358 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:48:00.844358 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application with Lock( Oct 10 11:48:00.844358 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:48:00.844358 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:48:00.844358 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:48:00.844358 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:48:00.844358 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:48:00.844358 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:48:00.844358 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:48:00.844358 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:48:00.846106 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 10 11:48:00.846106 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 10 11:48:00.846106 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 10 11:48:00.846106 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 10 11:48:00.846106 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 10 11:48:00.846106 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 10 11:48:00.846106 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 10 11:48:00.846106 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Oct 10 11:48:00.846106 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 478/985] 10.222.0.46 () {56 vars in 1111 bytes} [Mon Oct 10 11:48:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 10 11:48:00.851212 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7baa28db-75ed-4611-b38b-21f1348601df None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:48:00.855177 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-764305c1-27ba-49d9-88be-52ccb786b56a None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:48:00.856724 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-764305c1-27ba-49d9-88be-52ccb786b56a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:48:00.949793 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 508/986] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 10 11:48:00 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 10 11:48:00.976309 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7baa28db-75ed-4611-b38b-21f1348601df None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:48:01.019860 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62005736-112d-4a4b-8289-dbd661f1ad03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:48:01.020506 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62005736-112d-4a4b-8289-dbd661f1ad03 None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:48:01.020786 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62005736-112d-4a4b-8289-dbd661f1ad03 None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:48:01.021063 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62005736-112d-4a4b-8289-dbd661f1ad03 None admin] PATH_INFO: `/v3/projects` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:48:01.021800 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62005736-112d-4a4b-8289-dbd661f1ad03 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:48:01.022109 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62005736-112d-4a4b-8289-dbd661f1ad03 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:48:01.023676 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62005736-112d-4a4b-8289-dbd661f1ad03 None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:48:01.046708 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 479/987] 10.222.0.46 () {56 vars in 1148 bytes} [Mon Oct 10 11:48:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 11:48:01.049729 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764305c1-27ba-49d9-88be-52ccb786b56a None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:48:01.227486 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:48:01.228475 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62005736-112d-4a4b-8289-dbd661f1ad03 None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:48:01.235052 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764305c1-27ba-49d9-88be-52ccb786b56a None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:48:01.276184 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:48:01.305326 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfab357-615d-488a-a387-466e993879d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:48:01.306820 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfab357-615d-488a-a387-466e993879d2 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:48:01.307937 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfab357-615d-488a-a387-466e993879d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:48:01.308441 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfab357-615d-488a-a387-466e993879d2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:48:01.309919 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdfab357-615d-488a-a387-466e993879d2 None admin] RBAC: Adding query filter params (role.id=66c72919d8744554bc2e7c3dcc683e1f, scope.project.id=4ad3772f7b6a47c28b7ba37a9316a3c1, user.id=d6430593eecf45d8b891c3b55de4a944) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:48:01.310498 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdfab357-615d-488a-a387-466e993879d2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:48:01.313435 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdfab357-615d-488a-a387-466e993879d2 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:48:01.326764 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 509/988] 10.222.0.46 () {56 vars in 1374 bytes} [Mon Oct 10 11:48:01 2022] GET /identity/v3/role_assignments?user.id=d6430593eecf45d8b891c3b55de4a944&scope.project.id=4ad3772f7b6a47c28b7ba37a9316a3c1&role.id=66c72919d8744554bc2e7c3dcc683e1f => generated 608 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:48:01.396998 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62005736-112d-4a4b-8289-dbd661f1ad03 None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:48:01.471527 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b84e1d-f628-428c-b600-aa1ab837638f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:48:01.472177 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b84e1d-f628-428c-b600-aa1ab837638f None admin] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:48:01.472466 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b84e1d-f628-428c-b600-aa1ab837638f None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:48:01.472886 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b84e1d-f628-428c-b600-aa1ab837638f None admin] PATH_INFO: `/v3/services` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:48:01.473558 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b84e1d-f628-428c-b600-aa1ab837638f None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:48:01.475037 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b84e1d-f628-428c-b600-aa1ab837638f None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:48:01.550594 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 480/989] 10.222.0.46 () {60 vars in 1141 bytes} [Mon Oct 10 11:48:01 2022] POST /identity/v3/services => generated 242 bytes in 320 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 10 11:48:02.818398 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94dba12-0314-46a1-96d0-ab8aa25b75d6 None None] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:48:02.819180 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94dba12-0314-46a1-96d0-ab8aa25b75d6 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:48:02.819180 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94dba12-0314-46a1-96d0-ab8aa25b75d6 None None] PATH_INFO: `/` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:48:02.820593 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 510/990] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 10 11:48:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:48:02.829383 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2895d0f3-5b89-409b-9423-a96cd2505fa4 None None] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:48:02.830176 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2895d0f3-5b89-409b-9423-a96cd2505fa4 None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:48:02.830176 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2895d0f3-5b89-409b-9423-a96cd2505fa4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:48:02.859218 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.auth.core [None req-2895d0f3-5b89-409b-9423-a96cd2505fa4 None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:48:02.861970 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-2895d0f3-5b89-409b-9423-a96cd2505fa4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:48:02.959061 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 481/991] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 10 11:48:02 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 10 11:48:03.232837 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94dba12-0314-46a1-96d0-ab8aa25b75d6 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:48:03.395747 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94dba12-0314-46a1-96d0-ab8aa25b75d6 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:48:03.454579 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9e64b5-2e18-43b8-8112-7879b994d1af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:48:03.455701 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9e64b5-2e18-43b8-8112-7879b994d1af None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:48:03.456262 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9e64b5-2e18-43b8-8112-7879b994d1af None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:48:03.456832 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9e64b5-2e18-43b8-8112-7879b994d1af None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:48:03.457930 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9e64b5-2e18-43b8-8112-7879b994d1af None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:48:03.460354 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9e64b5-2e18-43b8-8112-7879b994d1af None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:48:03.480424 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application [None req-3a9e64b5-2e18-43b8-8112-7879b994d1af None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 10 11:48:03.480424 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:48:03.480424 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 10 11:48:03.480424 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 10 11:48:03.480424 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 10 11:48:03.480424 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 10 11:48:03.480424 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 10 11:48:03.480424 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 10 11:48:03.480424 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 10 11:48:03.480424 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Oct 10 11:48:03.480424 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 10 11:48:03.480424 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Oct 10 11:48:03.480424 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:48:03.480424 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:48:03.480424 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:48:03.480424 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:48:03.480424 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:48:03.481472 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:48:03.481472 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:48:03.481472 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:48:03.481472 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:48:03.481472 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:48:03.481472 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:48:03.481472 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 10 11:48:03.481472 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 10 11:48:03.481472 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 10 11:48:03.481472 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 10 11:48:03.481472 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:48:03.481472 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:48:03.481472 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:48:03.481472 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:48:03.481472 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:48:03.481472 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:48:03.482335 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:48:03.482335 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application with Lock( Oct 10 11:48:03.482335 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:48:03.482335 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:48:03.482335 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:48:03.482335 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:48:03.482335 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:48:03.482335 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:48:03.482335 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:48:03.482335 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:48:03.482335 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 10 11:48:03.482335 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 10 11:48:03.482335 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 10 11:48:03.482335 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Oct 10 11:48:03.495811 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 511/992] 10.222.0.46 () {56 vars in 1126 bytes} [Mon Oct 10 11:48:03 2022] GET /identity/v3/services/object-store => generated 93 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:48:03.496836 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2895d0f3-5b89-409b-9423-a96cd2505fa4 None None] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:48:03.525288 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:48:03.613374 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2895d0f3-5b89-409b-9423-a96cd2505fa4 None None] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:48:03.653934 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85806992-c34c-4595-94ac-a03c2b9de0f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:48:03.654617 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85806992-c34c-4595-94ac-a03c2b9de0f0 None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:48:03.654928 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85806992-c34c-4595-94ac-a03c2b9de0f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:48:03.655204 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85806992-c34c-4595-94ac-a03c2b9de0f0 None admin] PATH_INFO: `/v3/services` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:48:03.655853 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85806992-c34c-4595-94ac-a03c2b9de0f0 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:48:03.656160 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85806992-c34c-4595-94ac-a03c2b9de0f0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:48:03.657631 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85806992-c34c-4595-94ac-a03c2b9de0f0 None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:48:03.667248 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 482/993] 10.222.0.46 () {56 vars in 1122 bytes} [Mon Oct 10 11:48:03 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:48:03.670044 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9e64b5-2e18-43b8-8112-7879b994d1af None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:48:03.717072 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:48:03.808331 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9e64b5-2e18-43b8-8112-7879b994d1af None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:48:03.849031 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7026b45f-8675-41f0-a222-3fd9bd0050f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:48:03.850052 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7026b45f-8675-41f0-a222-3fd9bd0050f3 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:48:03.850052 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7026b45f-8675-41f0-a222-3fd9bd0050f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:48:03.850315 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7026b45f-8675-41f0-a222-3fd9bd0050f3 None admin] PATH_INFO: `/v3/services` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:48:03.850855 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7026b45f-8675-41f0-a222-3fd9bd0050f3 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:48:03.851165 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7026b45f-8675-41f0-a222-3fd9bd0050f3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:48:03.852563 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7026b45f-8675-41f0-a222-3fd9bd0050f3 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:48:03.863414 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 512/994] 10.222.0.46 () {56 vars in 1122 bytes} [Mon Oct 10 11:48:03 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:48:03.865499 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85806992-c34c-4595-94ac-a03c2b9de0f0 None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:48:03.963270 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:48:03.982812 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85806992-c34c-4595-94ac-a03c2b9de0f0 None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:48:04.025596 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad193c41-3ffe-455e-acac-8060ffae542d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:48:04.026257 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad193c41-3ffe-455e-acac-8060ffae542d None admin] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:48:04.026538 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad193c41-3ffe-455e-acac-8060ffae542d None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:48:04.026853 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad193c41-3ffe-455e-acac-8060ffae542d None admin] PATH_INFO: `/v3/endpoints` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:48:04.027551 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad193c41-3ffe-455e-acac-8060ffae542d None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:48:04.028936 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad193c41-3ffe-455e-acac-8060ffae542d None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:48:04.091295 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20eb55da-e597-4db9-b8dd-aa26d3e28058 None None] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:48:04.091502 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20eb55da-e597-4db9-b8dd-aa26d3e28058 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:48:04.091771 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20eb55da-e597-4db9-b8dd-aa26d3e28058 None None] PATH_INFO: `/` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:48:04.093244 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 513/995] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 10 11:48:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:48:04.098923 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327298fe-805f-4c90-b2ad-3f01948fbe61 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:48:04.099306 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327298fe-805f-4c90-b2ad-3f01948fbe61 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:48:04.099621 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327298fe-805f-4c90-b2ad-3f01948fbe61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:48:04.117542 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 483/996] 10.222.0.46 () {60 vars in 1144 bytes} [Mon Oct 10 11:48:03 2022] POST /identity/v3/endpoints => generated 325 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 10 11:48:04.120790 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-327298fe-805f-4c90-b2ad-3f01948fbe61 None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:48:04.123004 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-327298fe-805f-4c90-b2ad-3f01948fbe61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:48:04.272745 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 514/997] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 10 11:48:04 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 10 11:48:04.294964 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040db9d4-3d23-4565-9c6b-04f8041ad2be None None] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:48:04.295492 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040db9d4-3d23-4565-9c6b-04f8041ad2be None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:48:04.295991 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040db9d4-3d23-4565-9c6b-04f8041ad2be None None] PATH_INFO: `/` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:48:04.298777 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 484/998] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 10 11:48:04 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:48:04.308105 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f3d91e-70d6-424e-b799-ff96e975c77c None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:48:04.308603 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f3d91e-70d6-424e-b799-ff96e975c77c None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:48:04.309088 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f3d91e-70d6-424e-b799-ff96e975c77c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:48:04.339616 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-92f3d91e-70d6-424e-b799-ff96e975c77c None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:48:04.341648 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-92f3d91e-70d6-424e-b799-ff96e975c77c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:48:04.363571 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:48:04.472256 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 515/999] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 10 11:48:04 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 10 11:48:04.738899 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040db9d4-3d23-4565-9c6b-04f8041ad2be None None] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:48:04.865964 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040db9d4-3d23-4565-9c6b-04f8041ad2be None None] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:48:04.904623 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4709ee1d-6d52-4dec-8a0f-b8965c1d8684 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:48:04.905274 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4709ee1d-6d52-4dec-8a0f-b8965c1d8684 None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:48:04.905564 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4709ee1d-6d52-4dec-8a0f-b8965c1d8684 None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:48:04.905813 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4709ee1d-6d52-4dec-8a0f-b8965c1d8684 None admin] PATH_INFO: `/v3/services/image` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:48:04.906421 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4709ee1d-6d52-4dec-8a0f-b8965c1d8684 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:48:04.907835 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4709ee1d-6d52-4dec-8a0f-b8965c1d8684 None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:48:04.918040 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application [None req-4709ee1d-6d52-4dec-8a0f-b8965c1d8684 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Oct 10 11:48:04.918040 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:48:04.918040 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 10 11:48:04.918040 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 10 11:48:04.918040 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 10 11:48:04.918040 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 10 11:48:04.918040 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 10 11:48:04.918040 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 10 11:48:04.918040 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 10 11:48:04.918040 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Oct 10 11:48:04.918040 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 10 11:48:04.918040 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Oct 10 11:48:04.918040 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:48:04.918040 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:48:04.918040 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:48:04.918040 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:48:04.918040 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:48:04.919893 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:48:04.919893 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:48:04.919893 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:48:04.919893 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:48:04.919893 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:48:04.919893 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:48:04.919893 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 10 11:48:04.919893 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 10 11:48:04.919893 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 10 11:48:04.919893 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 10 11:48:04.919893 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:48:04.919893 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:48:04.919893 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:48:04.919893 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:48:04.919893 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:48:04.919893 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:48:04.920804 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:48:04.920804 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application with Lock( Oct 10 11:48:04.920804 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:48:04.920804 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:48:04.920804 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:48:04.920804 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:48:04.920804 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:48:04.920804 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:48:04.920804 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:48:04.920804 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:48:04.920804 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 10 11:48:04.920804 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 10 11:48:04.920804 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 10 11:48:04.920804 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Oct 10 11:48:04.923129 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 485/1000] 10.222.0.46 () {56 vars in 1105 bytes} [Mon Oct 10 11:48:04 2022] GET /identity/v3/services/image => generated 86 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:48:04.925362 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f3d91e-70d6-424e-b799-ff96e975c77c None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:48:04.933997 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4709ee1d-6d52-4dec-8a0f-b8965c1d8684 None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:48:04.949136 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:48:05.052821 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f3d91e-70d6-424e-b799-ff96e975c77c None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:48:05.074424 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4709ee1d-6d52-4dec-8a0f-b8965c1d8684 None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:48:05.100920 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d20e7ea-60e2-4fb7-939a-2c4b0d56e293 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:48:05.101899 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d20e7ea-60e2-4fb7-939a-2c4b0d56e293 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:48:05.102283 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d20e7ea-60e2-4fb7-939a-2c4b0d56e293 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:48:05.102498 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d20e7ea-60e2-4fb7-939a-2c4b0d56e293 None admin] PATH_INFO: `/v3/services` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:48:05.103364 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d20e7ea-60e2-4fb7-939a-2c4b0d56e293 None admin] RBAC: Adding query filter params (name=image) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:48:05.103857 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d20e7ea-60e2-4fb7-939a-2c4b0d56e293 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:48:05.106871 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d20e7ea-60e2-4fb7-939a-2c4b0d56e293 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:48:05.126916 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 516/1001] 10.222.0.46 () {56 vars in 1108 bytes} [Mon Oct 10 11:48:04 2022] GET /identity/v3/services?name=image => generated 122 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:48:05.130715 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed19632-818f-448d-8ab7-274a4cec79ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:48:05.131738 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed19632-818f-448d-8ab7-274a4cec79ff None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:48:05.132189 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed19632-818f-448d-8ab7-274a4cec79ff None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:48:05.132607 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d20e7ea-60e2-4fb7-939a-2c4b0d56e293 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:48:05.132784 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed19632-818f-448d-8ab7-274a4cec79ff None admin] PATH_INFO: `/v3/services/compute` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:48:05.133491 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed19632-818f-448d-8ab7-274a4cec79ff None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:48:05.135961 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed19632-818f-448d-8ab7-274a4cec79ff None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:48:05.151579 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application [None req-4ed19632-818f-448d-8ab7-274a4cec79ff None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Oct 10 11:48:05.151579 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:48:05.151579 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 10 11:48:05.151579 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 10 11:48:05.151579 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 10 11:48:05.151579 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 10 11:48:05.151579 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 10 11:48:05.151579 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 10 11:48:05.151579 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 10 11:48:05.151579 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Oct 10 11:48:05.151579 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 10 11:48:05.151579 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Oct 10 11:48:05.151579 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:48:05.151579 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:48:05.151579 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:48:05.151579 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:48:05.151579 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:48:05.155601 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:48:05.155601 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:48:05.155601 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:48:05.155601 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:48:05.155601 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:48:05.155601 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:48:05.155601 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 10 11:48:05.155601 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 10 11:48:05.155601 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 10 11:48:05.155601 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 10 11:48:05.155601 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:48:05.155601 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:48:05.155601 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:48:05.155601 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:48:05.155601 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:48:05.155601 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:48:05.156765 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:48:05.156765 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application with Lock( Oct 10 11:48:05.156765 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:48:05.156765 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:48:05.156765 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:48:05.156765 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:48:05.156765 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:48:05.156765 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:48:05.156765 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:48:05.156765 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:48:05.156765 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 10 11:48:05.156765 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 10 11:48:05.156765 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 10 11:48:05.156765 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Oct 10 11:48:05.156765 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 486/1002] 10.222.0.46 () {56 vars in 1111 bytes} [Mon Oct 10 11:48:04 2022] GET /identity/v3/services/compute => generated 88 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:48:05.157870 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:48:05.160266 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed19632-818f-448d-8ab7-274a4cec79ff None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:48:05.253087 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:48:05.280773 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed19632-818f-448d-8ab7-274a4cec79ff None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:48:05.295811 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d20e7ea-60e2-4fb7-939a-2c4b0d56e293 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:48:05.325263 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3aececc-e767-4bf0-aedf-bf4d9988fb5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:48:05.325785 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3aececc-e767-4bf0-aedf-bf4d9988fb5e None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:48:05.326074 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3aececc-e767-4bf0-aedf-bf4d9988fb5e None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:48:05.326315 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3aececc-e767-4bf0-aedf-bf4d9988fb5e None admin] PATH_INFO: `/v3/services` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:48:05.326984 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3aececc-e767-4bf0-aedf-bf4d9988fb5e None admin] RBAC: Adding query filter params (name=compute) {{(pid=81676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:48:05.327286 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3aececc-e767-4bf0-aedf-bf4d9988fb5e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:48:05.328658 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3aececc-e767-4bf0-aedf-bf4d9988fb5e None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:48:05.338806 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 487/1003] 10.222.0.46 () {56 vars in 1112 bytes} [Mon Oct 10 11:48:05 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:48:05.339740 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3aececc-e767-4bf0-aedf-bf4d9988fb5e None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:48:05.345135 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738c010a-5556-4e17-a3e3-ebd50a8a3db6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:48:05.345767 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738c010a-5556-4e17-a3e3-ebd50a8a3db6 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:48:05.346047 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738c010a-5556-4e17-a3e3-ebd50a8a3db6 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:48:05.346323 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738c010a-5556-4e17-a3e3-ebd50a8a3db6 None admin] PATH_INFO: `/v3/services` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:48:05.347044 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738c010a-5556-4e17-a3e3-ebd50a8a3db6 None admin] RBAC: Adding query filter params (type=image) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:48:05.347355 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738c010a-5556-4e17-a3e3-ebd50a8a3db6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:48:05.348725 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738c010a-5556-4e17-a3e3-ebd50a8a3db6 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:48:05.354789 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 517/1004] 10.222.0.46 () {56 vars in 1108 bytes} [Mon Oct 10 11:48:05 2022] GET /identity/v3/services?type=image => generated 122 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 11:48:05.516137 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3aececc-e767-4bf0-aedf-bf4d9988fb5e None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:48:05.561012 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda7c08b-ae25-40a6-b327-bc45cebfab6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:48:05.561767 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda7c08b-ae25-40a6-b327-bc45cebfab6c None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:48:05.562115 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda7c08b-ae25-40a6-b327-bc45cebfab6c None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:48:05.562331 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda7c08b-ae25-40a6-b327-bc45cebfab6c None admin] PATH_INFO: `/v3/services` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:48:05.563288 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda7c08b-ae25-40a6-b327-bc45cebfab6c None admin] RBAC: Adding query filter params (type=compute) {{(pid=81676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:48:05.563719 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda7c08b-ae25-40a6-b327-bc45cebfab6c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:48:05.565502 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda7c08b-ae25-40a6-b327-bc45cebfab6c None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:48:05.571718 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 488/1005] 10.222.0.46 () {56 vars in 1112 bytes} [Mon Oct 10 11:48:05 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:48:05.580911 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738c010a-5556-4e17-a3e3-ebd50a8a3db6 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:48:05.673255 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:48:05.701337 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738c010a-5556-4e17-a3e3-ebd50a8a3db6 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:48:05.738594 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8edd4adb-763c-42c5-8023-83341205b631 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:48:05.739155 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edd4adb-763c-42c5-8023-83341205b631 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:48:05.739492 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edd4adb-763c-42c5-8023-83341205b631 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:48:05.739721 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edd4adb-763c-42c5-8023-83341205b631 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:48:05.740413 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8edd4adb-763c-42c5-8023-83341205b631 None admin] RBAC: Adding query filter params (interface=public, service_id=e1a77cb9eb254fa4962bb096ddeea009, region_id=RegionOne) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:48:05.740798 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8edd4adb-763c-42c5-8023-83341205b631 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:48:05.742126 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8edd4adb-763c-42c5-8023-83341205b631 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:48:05.749134 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 518/1006] 10.222.0.46 () {56 vars in 1251 bytes} [Mon Oct 10 11:48:05 2022] GET /identity/v3/endpoints?service_id=e1a77cb9eb254fa4962bb096ddeea009&interface=public®ion_id=RegionOne => generated 194 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:48:05.755024 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda7c08b-ae25-40a6-b327-bc45cebfab6c None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:48:05.758086 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:48:05.915768 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda7c08b-ae25-40a6-b327-bc45cebfab6c None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:48:05.956801 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51af5bc0-b989-458d-a868-7b1e74b894b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:48:05.957468 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51af5bc0-b989-458d-a868-7b1e74b894b1 None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:48:05.957755 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51af5bc0-b989-458d-a868-7b1e74b894b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:48:05.958032 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51af5bc0-b989-458d-a868-7b1e74b894b1 None admin] PATH_INFO: `/v3/services` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:48:05.958718 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51af5bc0-b989-458d-a868-7b1e74b894b1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:48:05.960133 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51af5bc0-b989-458d-a868-7b1e74b894b1 None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:48:05.967531 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 489/1007] 10.222.0.46 () {56 vars in 1087 bytes} [Mon Oct 10 11:48:05 2022] GET /identity/v3/services => generated 1727 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 10 11:48:06.435274 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2b61fa-1e6c-493d-97e4-ee6a75a6a22d None None] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:48:06.435806 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2b61fa-1e6c-493d-97e4-ee6a75a6a22d None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:48:06.436287 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2b61fa-1e6c-493d-97e4-ee6a75a6a22d None None] PATH_INFO: `/` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:48:06.438876 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 519/1008] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 10 11:48:06 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:48:06.444860 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2358d12b-95d3-4cf7-ba08-6431db5e11b0 None None] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:48:06.445237 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2358d12b-95d3-4cf7-ba08-6431db5e11b0 None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:48:06.445439 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2358d12b-95d3-4cf7-ba08-6431db5e11b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:48:06.466064 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.auth.core [None req-2358d12b-95d3-4cf7-ba08-6431db5e11b0 None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:48:06.467340 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-2358d12b-95d3-4cf7-ba08-6431db5e11b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:48:06.542472 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:48:06.551925 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 490/1009] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 10 11:48:06 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 10 11:48:06.827536 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2b61fa-1e6c-493d-97e4-ee6a75a6a22d None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:48:07.017585 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2b61fa-1e6c-493d-97e4-ee6a75a6a22d None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:48:07.084439 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457f23cf-4766-4749-89ae-8705d2c76760 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:48:07.085429 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457f23cf-4766-4749-89ae-8705d2c76760 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:48:07.085934 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457f23cf-4766-4749-89ae-8705d2c76760 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:48:07.086426 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457f23cf-4766-4749-89ae-8705d2c76760 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:48:07.089072 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457f23cf-4766-4749-89ae-8705d2c76760 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:48:07.091831 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457f23cf-4766-4749-89ae-8705d2c76760 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:48:07.095153 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 520/1010] 10.222.0.46 () {56 vars in 1108 bytes} [Mon Oct 10 11:48:06 2022] GET /identity/v3/domains/default => generated 199 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:48:07.104613 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2358d12b-95d3-4cf7-ba08-6431db5e11b0 None None] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:48:07.303152 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2358d12b-95d3-4cf7-ba08-6431db5e11b0 None None] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:48:07.376806 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3ba58b-25be-46af-ab7b-e0e5cdbd43a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:48:07.377888 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3ba58b-25be-46af-ab7b-e0e5cdbd43a5 None admin] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:48:07.378399 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3ba58b-25be-46af-ab7b-e0e5cdbd43a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:48:07.378949 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3ba58b-25be-46af-ab7b-e0e5cdbd43a5 None admin] PATH_INFO: `/v3/projects` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:48:07.380217 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3ba58b-25be-46af-ab7b-e0e5cdbd43a5 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:48:07.383109 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3ba58b-25be-46af-ab7b-e0e5cdbd43a5 None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:48:07.498285 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 491/1011] 10.222.0.46 () {60 vars in 1141 bytes} [Mon Oct 10 11:48:07 2022] POST /identity/v3/projects => generated 311 bytes in 395 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 11:48:07.630060 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b3c1a4-f63b-4c55-be32-10393d183420 None None] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:48:07.630588 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b3c1a4-f63b-4c55-be32-10393d183420 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:48:07.631019 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b3c1a4-f63b-4c55-be32-10393d183420 None None] PATH_INFO: `/` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:48:07.633727 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 521/1012] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 10 11:48:07 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:48:07.639918 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50d349c-4d39-4d0a-8289-0a3257a1235e None None] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:48:07.640177 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50d349c-4d39-4d0a-8289-0a3257a1235e None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:48:07.640550 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50d349c-4d39-4d0a-8289-0a3257a1235e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:48:07.661470 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.auth.core [None req-f50d349c-4d39-4d0a-8289-0a3257a1235e None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:48:07.662731 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-f50d349c-4d39-4d0a-8289-0a3257a1235e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:48:07.745777 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:48:07.784713 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 492/1013] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 10 11:48:07 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 10 11:48:08.064446 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b3c1a4-f63b-4c55-be32-10393d183420 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:48:08.074143 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:48:08.220925 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b3c1a4-f63b-4c55-be32-10393d183420 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:48:08.257038 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0592e20-ba7c-43bf-96c4-88d6150dc13c None None] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:48:08.257605 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0592e20-ba7c-43bf-96c4-88d6150dc13c None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:48:08.257605 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0592e20-ba7c-43bf-96c4-88d6150dc13c None None] PATH_INFO: `/` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:48:08.259047 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 493/1014] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 10 11:48:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 10 11:48:08.265409 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4497e061-df85-4fbb-acec-65bb7a1c40c8 None None] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:48:08.265789 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4497e061-df85-4fbb-acec-65bb7a1c40c8 None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:48:08.266140 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4497e061-df85-4fbb-acec-65bb7a1c40c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:48:08.266920 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32668dfc-a915-497e-9913-b09c86d1fc88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:48:08.267726 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32668dfc-a915-497e-9913-b09c86d1fc88 None admin] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:48:08.268150 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32668dfc-a915-497e-9913-b09c86d1fc88 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:48:08.268565 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32668dfc-a915-497e-9913-b09c86d1fc88 None admin] PATH_INFO: `/v3/services` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:48:08.269643 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32668dfc-a915-497e-9913-b09c86d1fc88 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:48:08.271435 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32668dfc-a915-497e-9913-b09c86d1fc88 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:48:08.287545 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.auth.core [None req-4497e061-df85-4fbb-acec-65bb7a1c40c8 None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:48:08.288774 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-4497e061-df85-4fbb-acec-65bb7a1c40c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:48:08.362291 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 494/1015] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 10 11:48:08 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 10 11:48:08.410184 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 522/1016] 10.222.0.46 () {60 vars in 1141 bytes} [Mon Oct 10 11:48:08 2022] POST /identity/v3/services => generated 242 bytes in 346 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 10 11:48:08.643676 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4497e061-df85-4fbb-acec-65bb7a1c40c8 None None] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:48:08.791920 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4497e061-df85-4fbb-acec-65bb7a1c40c8 None None] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:48:08.803052 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:48:08.842891 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed927f6f-dcdb-46c4-bf6f-556e93b9059f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:48:08.843738 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed927f6f-dcdb-46c4-bf6f-556e93b9059f None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:48:08.844149 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed927f6f-dcdb-46c4-bf6f-556e93b9059f None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:48:08.844609 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed927f6f-dcdb-46c4-bf6f-556e93b9059f None admin] PATH_INFO: `/v3/services/compute` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:48:08.845443 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed927f6f-dcdb-46c4-bf6f-556e93b9059f None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:48:08.847044 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed927f6f-dcdb-46c4-bf6f-556e93b9059f None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:48:08.858150 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application [None req-ed927f6f-dcdb-46c4-bf6f-556e93b9059f None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Oct 10 11:48:08.858150 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:48:08.858150 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 10 11:48:08.858150 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 10 11:48:08.858150 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 10 11:48:08.858150 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 10 11:48:08.858150 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 10 11:48:08.858150 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 10 11:48:08.858150 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 10 11:48:08.858150 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Oct 10 11:48:08.858150 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 10 11:48:08.858150 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Oct 10 11:48:08.858150 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:48:08.858150 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:48:08.858150 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:48:08.858150 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:48:08.858150 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:48:08.859504 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:48:08.859504 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:48:08.859504 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:48:08.859504 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:48:08.859504 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:48:08.859504 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:48:08.859504 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 10 11:48:08.859504 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 10 11:48:08.859504 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 10 11:48:08.859504 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 10 11:48:08.859504 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:48:08.859504 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:48:08.859504 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:48:08.859504 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:48:08.859504 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:48:08.859504 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:48:08.860347 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:48:08.860347 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application with Lock( Oct 10 11:48:08.860347 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:48:08.860347 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:48:08.860347 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:48:08.860347 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:48:08.860347 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:48:08.860347 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:48:08.860347 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:48:08.860347 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:48:08.860347 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 10 11:48:08.860347 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 10 11:48:08.860347 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 10 11:48:08.860347 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Oct 10 11:48:08.860347 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 495/1017] 10.222.0.46 () {56 vars in 1111 bytes} [Mon Oct 10 11:48:08 2022] GET /identity/v3/services/compute => generated 88 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 10 11:48:08.868401 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32668dfc-a915-497e-9913-b09c86d1fc88 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:48:08.985175 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:48:09.027024 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32668dfc-a915-497e-9913-b09c86d1fc88 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:48:09.069227 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ef6710-803f-4e88-a898-4538e453861b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:48:09.069227 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ef6710-803f-4e88-a898-4538e453861b None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:48:09.069227 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ef6710-803f-4e88-a898-4538e453861b None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:48:09.069227 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ef6710-803f-4e88-a898-4538e453861b None admin] PATH_INFO: `/v3/services` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:48:09.069820 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ef6710-803f-4e88-a898-4538e453861b None admin] RBAC: Adding query filter params (name=compute) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:48:09.069820 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ef6710-803f-4e88-a898-4538e453861b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:48:09.069820 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ef6710-803f-4e88-a898-4538e453861b None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:48:09.076005 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 523/1018] 10.222.0.46 () {56 vars in 1112 bytes} [Mon Oct 10 11:48:08 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:48:09.082341 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed927f6f-dcdb-46c4-bf6f-556e93b9059f None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:48:09.212002 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed927f6f-dcdb-46c4-bf6f-556e93b9059f None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:48:09.252738 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868ad482-c36f-4553-a72c-75be283dfb32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:48:09.253285 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868ad482-c36f-4553-a72c-75be283dfb32 None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:48:09.253568 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868ad482-c36f-4553-a72c-75be283dfb32 None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:48:09.253846 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868ad482-c36f-4553-a72c-75be283dfb32 None admin] PATH_INFO: `/v3/services` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:48:09.254498 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868ad482-c36f-4553-a72c-75be283dfb32 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:48:09.254845 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868ad482-c36f-4553-a72c-75be283dfb32 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:48:09.256276 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868ad482-c36f-4553-a72c-75be283dfb32 None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:48:09.262089 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 496/1019] 10.222.0.46 () {56 vars in 1112 bytes} [Mon Oct 10 11:48:09 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:48:09.268751 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ef6710-803f-4e88-a898-4538e453861b None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:48:09.466715 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ef6710-803f-4e88-a898-4538e453861b None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:48:09.524204 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:48:09.533765 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4596b5b5-1392-4f37-afdb-61a25492032f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:48:09.534828 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4596b5b5-1392-4f37-afdb-61a25492032f None admin] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:48:09.535310 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4596b5b5-1392-4f37-afdb-61a25492032f None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:48:09.537399 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4596b5b5-1392-4f37-afdb-61a25492032f None admin] PATH_INFO: `/v3/endpoints` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:48:09.537399 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4596b5b5-1392-4f37-afdb-61a25492032f None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:48:09.539281 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4596b5b5-1392-4f37-afdb-61a25492032f None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:48:09.642708 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 524/1020] 10.222.0.46 () {60 vars in 1144 bytes} [Mon Oct 10 11:48:09 2022] POST /identity/v3/endpoints => generated 334 bytes in 371 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 10 11:48:10.331850 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec594e6-6729-4710-8c0e-034aaf9c08e0 None None] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:48:10.332417 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec594e6-6729-4710-8c0e-034aaf9c08e0 None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:48:10.332844 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec594e6-6729-4710-8c0e-034aaf9c08e0 None None] PATH_INFO: `/` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:48:10.335407 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 497/1021] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 10 11:48:10 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:48:10.341799 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7d9fce-94e3-42ce-85bf-0354aeefc05f None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:48:10.342553 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7d9fce-94e3-42ce-85bf-0354aeefc05f None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:48:10.342553 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7d9fce-94e3-42ce-85bf-0354aeefc05f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:48:10.362854 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-0e7d9fce-94e3-42ce-85bf-0354aeefc05f None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:48:10.364065 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-0e7d9fce-94e3-42ce-85bf-0354aeefc05f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:48:10.498727 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 525/1022] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 10 11:48:10 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 10 11:48:10.817062 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9345c24-d7d5-4a60-b6b2-ca0e3d2e4f53 None None] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:48:10.817729 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9345c24-d7d5-4a60-b6b2-ca0e3d2e4f53 None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:48:10.818226 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9345c24-d7d5-4a60-b6b2-ca0e3d2e4f53 None None] PATH_INFO: `/` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:48:10.820883 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 498/1023] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 10 11:48:10 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:48:10.827689 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d618c2e5-2b2f-4d00-8ce2-783bbaff5ffc None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:48:10.828014 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d618c2e5-2b2f-4d00-8ce2-783bbaff5ffc None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:48:10.828349 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d618c2e5-2b2f-4d00-8ce2-783bbaff5ffc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:48:10.849527 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-d618c2e5-2b2f-4d00-8ce2-783bbaff5ffc None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:48:10.850817 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-d618c2e5-2b2f-4d00-8ce2-783bbaff5ffc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:48:10.936157 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 526/1024] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 10 11:48:10 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 10 11:48:11.011833 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9345c24-d7d5-4a60-b6b2-ca0e3d2e4f53 None None] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:48:11.071988 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:48:11.209149 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d618c2e5-2b2f-4d00-8ce2-783bbaff5ffc None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:48:11.210325 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9345c24-d7d5-4a60-b6b2-ca0e3d2e4f53 None None] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:48:11.260379 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:48:11.266361 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fced637c-6050-496b-99a4-48e96042ce36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:48:11.267041 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fced637c-6050-496b-99a4-48e96042ce36 None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:48:11.267318 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fced637c-6050-496b-99a4-48e96042ce36 None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:48:11.267591 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fced637c-6050-496b-99a4-48e96042ce36 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:48:11.268779 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fced637c-6050-496b-99a4-48e96042ce36 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:48:11.270210 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fced637c-6050-496b-99a4-48e96042ce36 None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:48:11.271852 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 499/1025] 10.222.0.46 () {56 vars in 1108 bytes} [Mon Oct 10 11:48:11 2022] GET /identity/v3/domains/default => generated 199 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:48:11.281890 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fced637c-6050-496b-99a4-48e96042ce36 None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:48:11.326308 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d618c2e5-2b2f-4d00-8ce2-783bbaff5ffc None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:48:11.367134 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d2faae-a705-4e8b-8dbb-d65a0e2f38d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:48:11.368964 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d2faae-a705-4e8b-8dbb-d65a0e2f38d9 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:48:11.369508 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d2faae-a705-4e8b-8dbb-d65a0e2f38d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:48:11.369952 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d2faae-a705-4e8b-8dbb-d65a0e2f38d9 None admin] PATH_INFO: `/v3/services/image` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:48:11.370704 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d2faae-a705-4e8b-8dbb-d65a0e2f38d9 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:48:11.372319 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d2faae-a705-4e8b-8dbb-d65a0e2f38d9 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:48:11.381863 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application [None req-f9d2faae-a705-4e8b-8dbb-d65a0e2f38d9 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Oct 10 11:48:11.381863 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:48:11.381863 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 10 11:48:11.381863 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 10 11:48:11.381863 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 10 11:48:11.381863 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 10 11:48:11.381863 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 10 11:48:11.381863 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 10 11:48:11.381863 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 10 11:48:11.381863 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Oct 10 11:48:11.381863 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 10 11:48:11.381863 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Oct 10 11:48:11.381863 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:48:11.381863 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:48:11.381863 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:48:11.381863 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:48:11.381863 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:48:11.382929 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:48:11.382929 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:48:11.382929 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:48:11.382929 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:48:11.382929 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:48:11.382929 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:48:11.382929 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 10 11:48:11.382929 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 10 11:48:11.382929 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 10 11:48:11.382929 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 10 11:48:11.382929 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:48:11.382929 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:48:11.382929 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:48:11.382929 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:48:11.382929 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:48:11.382929 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:48:11.387496 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:48:11.387496 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application with Lock( Oct 10 11:48:11.387496 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:48:11.387496 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:48:11.387496 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:48:11.387496 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:48:11.387496 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:48:11.387496 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:48:11.387496 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:48:11.387496 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:48:11.387496 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 10 11:48:11.387496 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 10 11:48:11.387496 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 10 11:48:11.387496 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Oct 10 11:48:11.387496 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 527/1026] 10.222.0.46 () {56 vars in 1105 bytes} [Mon Oct 10 11:48:11 2022] GET /identity/v3/services/image => generated 86 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:48:11.390392 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d2faae-a705-4e8b-8dbb-d65a0e2f38d9 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:48:11.442940 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:48:11.446843 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fced637c-6050-496b-99a4-48e96042ce36 None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:48:11.491772 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552987c7-1271-4858-8505-c3703974842d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:48:11.492301 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552987c7-1271-4858-8505-c3703974842d None admin] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:48:11.492521 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552987c7-1271-4858-8505-c3703974842d None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:48:11.493213 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552987c7-1271-4858-8505-c3703974842d None admin] PATH_INFO: `/v3/users` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:48:11.494071 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552987c7-1271-4858-8505-c3703974842d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:48:11.496265 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552987c7-1271-4858-8505-c3703974842d None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:48:11.529314 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d2faae-a705-4e8b-8dbb-d65a0e2f38d9 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:48:11.537326 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:48:11.565781 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9cd3653-a0c0-4697-be9e-f5063cb32327 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:48:11.566509 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cd3653-a0c0-4697-be9e-f5063cb32327 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:48:11.566881 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cd3653-a0c0-4697-be9e-f5063cb32327 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:48:11.567156 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cd3653-a0c0-4697-be9e-f5063cb32327 None admin] PATH_INFO: `/v3/services` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:48:11.567818 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9cd3653-a0c0-4697-be9e-f5063cb32327 None admin] RBAC: Adding query filter params (name=image) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:48:11.568122 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9cd3653-a0c0-4697-be9e-f5063cb32327 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:48:11.569490 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9cd3653-a0c0-4697-be9e-f5063cb32327 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:48:11.574803 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 528/1027] 10.222.0.46 () {56 vars in 1108 bytes} [Mon Oct 10 11:48:11 2022] GET /identity/v3/services?name=image => generated 122 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 11:48:11.580708 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9cd3653-a0c0-4697-be9e-f5063cb32327 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:48:11.620885 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 500/1028] 10.222.0.46 () {60 vars in 1132 bytes} [Mon Oct 10 11:48:11 2022] POST /identity/v3/users => generated 285 bytes in 340 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 10 11:48:11.729847 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9cd3653-a0c0-4697-be9e-f5063cb32327 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:48:11.787565 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b619e9d0-167c-405b-84e5-1f905b01fd70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:48:11.788550 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b619e9d0-167c-405b-84e5-1f905b01fd70 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:48:11.789001 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b619e9d0-167c-405b-84e5-1f905b01fd70 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:48:11.789453 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b619e9d0-167c-405b-84e5-1f905b01fd70 None admin] PATH_INFO: `/v3/services` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:48:11.790550 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b619e9d0-167c-405b-84e5-1f905b01fd70 None admin] RBAC: Adding query filter params (type=image) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:48:11.791840 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b619e9d0-167c-405b-84e5-1f905b01fd70 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:48:11.794297 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b619e9d0-167c-405b-84e5-1f905b01fd70 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:48:11.808512 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 529/1029] 10.222.0.46 () {56 vars in 1108 bytes} [Mon Oct 10 11:48:11 2022] GET /identity/v3/services?type=image => generated 350 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 11:48:11.810442 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552987c7-1271-4858-8505-c3703974842d None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:48:12.005814 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552987c7-1271-4858-8505-c3703974842d None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:48:12.063095 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c29ceb-aeb3-4ca7-b62f-b07153e56f65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:48:12.063712 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c29ceb-aeb3-4ca7-b62f-b07153e56f65 None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:48:12.063987 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c29ceb-aeb3-4ca7-b62f-b07153e56f65 None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:48:12.064256 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c29ceb-aeb3-4ca7-b62f-b07153e56f65 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:48:12.064970 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c29ceb-aeb3-4ca7-b62f-b07153e56f65 None admin] RBAC: Adding query filter params (interface=public, service_id=eee815a180b54c96bb0f15a2fe521310, region_id=RegionOne) {{(pid=81676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:48:12.065276 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c29ceb-aeb3-4ca7-b62f-b07153e56f65 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:48:12.066656 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c29ceb-aeb3-4ca7-b62f-b07153e56f65 None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:48:12.074294 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 501/1030] 10.222.0.46 () {56 vars in 1251 bytes} [Mon Oct 10 11:48:11 2022] GET /identity/v3/endpoints?service_id=eee815a180b54c96bb0f15a2fe521310&interface=public®ion_id=RegionOne => generated 194 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 11:48:12.080705 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b619e9d0-167c-405b-84e5-1f905b01fd70 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:48:12.145807 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:48:12.271326 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b619e9d0-167c-405b-84e5-1f905b01fd70 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:48:12.328996 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b5ebd5-476a-4089-975f-f293c7c4e413 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:48:12.330032 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b5ebd5-476a-4089-975f-f293c7c4e413 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:48:12.330523 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b5ebd5-476a-4089-975f-f293c7c4e413 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:48:12.331037 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b5ebd5-476a-4089-975f-f293c7c4e413 None admin] PATH_INFO: `/v3/services` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:48:12.332116 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b5ebd5-476a-4089-975f-f293c7c4e413 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:48:12.334611 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b5ebd5-476a-4089-975f-f293c7c4e413 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:48:12.353246 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 530/1031] 10.222.0.46 () {56 vars in 1087 bytes} [Mon Oct 10 11:48:12 2022] GET /identity/v3/services => generated 1957 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 10 11:48:12.639109 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59566eb9-2327-430f-9714-2ef39828e93a None None] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:48:12.639647 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59566eb9-2327-430f-9714-2ef39828e93a None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:48:12.639647 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59566eb9-2327-430f-9714-2ef39828e93a None None] PATH_INFO: `/` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:48:12.641044 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 502/1032] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 10 11:48:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:48:12.647019 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c95ea1b-22e1-4234-b0f7-9f0e620f8b2a None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:48:12.647563 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c95ea1b-22e1-4234-b0f7-9f0e620f8b2a None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:48:12.647563 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c95ea1b-22e1-4234-b0f7-9f0e620f8b2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:48:12.667880 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-7c95ea1b-22e1-4234-b0f7-9f0e620f8b2a None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:48:12.669113 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-7c95ea1b-22e1-4234-b0f7-9f0e620f8b2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:48:12.742483 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 531/1033] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 10 11:48:12 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 10 11:48:13.008001 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59566eb9-2327-430f-9714-2ef39828e93a None None] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:48:13.033600 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:48:13.124295 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59566eb9-2327-430f-9714-2ef39828e93a None None] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:48:13.161598 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59cf96da-3c90-4490-9897-3b889b412e47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:48:13.162248 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cf96da-3c90-4490-9897-3b889b412e47 None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:48:13.162536 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cf96da-3c90-4490-9897-3b889b412e47 None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:48:13.162829 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cf96da-3c90-4490-9897-3b889b412e47 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:48:13.174243 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59cf96da-3c90-4490-9897-3b889b412e47 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 10 11:48:13.179182 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application [None req-59cf96da-3c90-4490-9897-3b889b412e47 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 10 11:48:13.179182 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:48:13.179182 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:48:13.179182 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:48:13.179182 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:48:13.179182 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:48:13.179182 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:48:13.179182 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:48:13.179182 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:48:13.179182 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:48:13.179182 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:48:13.179182 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:48:13.179182 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 10 11:48:13.179182 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 10 11:48:13.179182 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 10 11:48:13.179182 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 10 11:48:13.180089 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 10 11:48:13.180089 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 10 11:48:13.180089 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 10 11:48:13.180089 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 10 11:48:13.180089 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:48:13.180089 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:48:13.180089 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:48:13.180089 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:48:13.180089 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:48:13.180089 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:48:13.180089 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:48:13.180089 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application with Lock( Oct 10 11:48:13.180089 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:48:13.180089 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:48:13.180089 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:48:13.180089 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:48:13.180935 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:48:13.180935 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:48:13.180935 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:48:13.180935 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:48:13.180935 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 10 11:48:13.180935 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 10 11:48:13.180935 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 10 11:48:13.180935 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 10 11:48:13.180935 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 10 11:48:13.180935 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 10 11:48:13.180935 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 10 11:48:13.180935 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Oct 10 11:48:13.180935 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 503/1034] 10.222.0.46 () {56 vars in 1120 bytes} [Mon Oct 10 11:48:13 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:48:13.188298 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c95ea1b-22e1-4234-b0f7-9f0e620f8b2a None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:48:13.217255 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:48:13.295643 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c95ea1b-22e1-4234-b0f7-9f0e620f8b2a None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:48:13.331188 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa13e1d-af61-4c57-947f-21ed4f405250 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:48:13.331819 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa13e1d-af61-4c57-947f-21ed4f405250 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:48:13.332099 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa13e1d-af61-4c57-947f-21ed4f405250 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:48:13.332374 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa13e1d-af61-4c57-947f-21ed4f405250 None admin] PATH_INFO: `/v3/roles` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:48:13.333052 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa13e1d-af61-4c57-947f-21ed4f405250 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:48:13.333354 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa13e1d-af61-4c57-947f-21ed4f405250 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:48:13.334752 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa13e1d-af61-4c57-947f-21ed4f405250 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:48:13.348738 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 532/1035] 10.222.0.46 () {56 vars in 1115 bytes} [Mon Oct 10 11:48:13 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:48:13.350017 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59cf96da-3c90-4490-9897-3b889b412e47 None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:48:13.436243 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:48:13.457623 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59cf96da-3c90-4490-9897-3b889b412e47 None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:48:13.500890 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543ceddd-9ef0-4273-aea3-9050a9ae756b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:48:13.501548 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543ceddd-9ef0-4273-aea3-9050a9ae756b None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:48:13.501827 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543ceddd-9ef0-4273-aea3-9050a9ae756b None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:48:13.502163 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543ceddd-9ef0-4273-aea3-9050a9ae756b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:48:13.513402 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543ceddd-9ef0-4273-aea3-9050a9ae756b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:48:13.514888 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543ceddd-9ef0-4273-aea3-9050a9ae756b None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:48:13.530332 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application [None req-543ceddd-9ef0-4273-aea3-9050a9ae756b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:48:13.530332 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:48:13.530332 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:48:13.530332 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:48:13.530332 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:48:13.530332 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:48:13.530332 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:48:13.530332 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:48:13.530332 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:48:13.530332 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:48:13.530332 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:48:13.530332 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:48:13.530332 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 10 11:48:13.530332 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 10 11:48:13.530332 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 10 11:48:13.530332 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 10 11:48:13.531434 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:48:13.531434 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:48:13.531434 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:48:13.531434 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:48:13.531434 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:48:13.531434 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:48:13.531434 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:48:13.531434 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application with Lock( Oct 10 11:48:13.531434 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:48:13.531434 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:48:13.531434 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:48:13.531434 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:48:13.531434 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:48:13.531434 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:48:13.531434 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:48:13.531434 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:48:13.531434 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 10 11:48:13.532338 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 10 11:48:13.532338 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:48:13.532338 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Oct 10 11:48:13.533083 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 504/1036] 10.222.0.46 () {56 vars in 1108 bytes} [Mon Oct 10 11:48:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 10 11:48:13.537335 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa13e1d-af61-4c57-947f-21ed4f405250 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:48:13.635812 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:48:13.645741 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa13e1d-af61-4c57-947f-21ed4f405250 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:48:13.684284 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a436ced-7dc2-4af3-b9f1-21a71d3bb177 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:48:13.684963 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a436ced-7dc2-4af3-b9f1-21a71d3bb177 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:48:13.685218 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a436ced-7dc2-4af3-b9f1-21a71d3bb177 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:48:13.685453 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a436ced-7dc2-4af3-b9f1-21a71d3bb177 None admin] PATH_INFO: `/v3/domains` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:48:13.686123 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a436ced-7dc2-4af3-b9f1-21a71d3bb177 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:48:13.686424 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a436ced-7dc2-4af3-b9f1-21a71d3bb177 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:48:13.687889 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a436ced-7dc2-4af3-b9f1-21a71d3bb177 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:48:13.702464 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 533/1037] 10.222.0.46 () {56 vars in 1109 bytes} [Mon Oct 10 11:48:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:48:13.708195 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543ceddd-9ef0-4273-aea3-9050a9ae756b None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:48:13.885530 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543ceddd-9ef0-4273-aea3-9050a9ae756b None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:48:13.941399 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:48:13.950923 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5da4d3-c97e-4db7-968d-330d51a75712 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:48:13.952047 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5da4d3-c97e-4db7-968d-330d51a75712 None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:48:13.952597 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5da4d3-c97e-4db7-968d-330d51a75712 None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:48:13.953044 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5da4d3-c97e-4db7-968d-330d51a75712 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:48:13.966968 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5da4d3-c97e-4db7-968d-330d51a75712 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:48:13.969702 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5da4d3-c97e-4db7-968d-330d51a75712 None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:48:13.991533 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application [None req-4b5da4d3-c97e-4db7-968d-330d51a75712 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 10 11:48:13.991533 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:48:13.991533 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:48:13.991533 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:48:13.991533 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:48:13.991533 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:48:13.991533 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:48:13.991533 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:48:13.991533 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:48:13.991533 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:48:13.991533 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:48:13.991533 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:48:13.991533 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 10 11:48:13.991533 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 10 11:48:13.991533 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 10 11:48:13.991533 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 10 11:48:13.992542 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:48:13.992542 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:48:13.992542 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 10 11:48:13.992542 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 10 11:48:13.992542 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 10 11:48:13.992542 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 10 11:48:13.992542 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:48:13.992542 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:48:13.992542 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:48:13.992542 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:48:13.992542 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:48:13.992542 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application with Lock( Oct 10 11:48:13.992542 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:48:13.992542 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:48:13.992542 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:48:13.992542 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:48:13.992542 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:48:13.992542 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:48:13.993761 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:48:13.993761 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:48:13.993761 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 10 11:48:13.993761 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 10 11:48:13.993761 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 10 11:48:13.993761 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 10 11:48:13.993761 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 10 11:48:13.993761 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 10 11:48:13.993761 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 10 11:48:13.993761 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Oct 10 11:48:13.995559 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 505/1038] 10.222.0.46 () {56 vars in 1093 bytes} [Mon Oct 10 11:48:13 2022] GET /identity/v3/users/nova => generated 82 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:48:14.004424 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a436ced-7dc2-4af3-b9f1-21a71d3bb177 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:48:14.187708 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a436ced-7dc2-4af3-b9f1-21a71d3bb177 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:48:14.250865 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5a5bf4-7015-4c74-a096-e3fc2c58cd94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:48:14.251831 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5a5bf4-7015-4c74-a096-e3fc2c58cd94 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:48:14.252318 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5a5bf4-7015-4c74-a096-e3fc2c58cd94 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:48:14.252800 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5a5bf4-7015-4c74-a096-e3fc2c58cd94 None admin] PATH_INFO: `/v3/users` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:48:14.254187 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5a5bf4-7015-4c74-a096-e3fc2c58cd94 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:48:14.254762 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5a5bf4-7015-4c74-a096-e3fc2c58cd94 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:48:14.257758 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5a5bf4-7015-4c74-a096-e3fc2c58cd94 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:48:14.298708 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 534/1039] 10.222.0.46 () {56 vars in 1133 bytes} [Mon Oct 10 11:48:14 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 11:48:14.303555 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5da4d3-c97e-4db7-968d-330d51a75712 None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:48:14.502008 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5da4d3-c97e-4db7-968d-330d51a75712 None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:48:14.569306 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30571a04-d50e-46c7-929b-30e054eac669 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:48:14.570344 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30571a04-d50e-46c7-929b-30e054eac669 None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:48:14.570882 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30571a04-d50e-46c7-929b-30e054eac669 None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:48:14.571378 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30571a04-d50e-46c7-929b-30e054eac669 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:48:14.688963 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:48:14.707456 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30571a04-d50e-46c7-929b-30e054eac669 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:48:14.707653 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ff2702-1e71-4ae6-8747-33000849e3af None None] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:48:14.708066 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ff2702-1e71-4ae6-8747-33000849e3af None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:48:14.708400 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ff2702-1e71-4ae6-8747-33000849e3af None None] PATH_INFO: `/` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:48:14.710109 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30571a04-d50e-46c7-929b-30e054eac669 None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:48:14.711084 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 535/1040] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 10 11:48:14 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 10 11:48:14.717530 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317623ab-1f61-4d26-bde5-31d349ee78f7 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:48:14.717914 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317623ab-1f61-4d26-bde5-31d349ee78f7 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:48:14.718070 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317623ab-1f61-4d26-bde5-31d349ee78f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:48:14.739284 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-317623ab-1f61-4d26-bde5-31d349ee78f7 None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:48:14.740492 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-317623ab-1f61-4d26-bde5-31d349ee78f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:48:14.743751 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:48:14.845993 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 536/1041] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 10 11:48:14 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 10 11:48:14.859482 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 506/1042] 10.222.0.46 () {58 vars in 1282 bytes} [Mon Oct 10 11:48:14 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 557 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Oct 10 11:48:14.866745 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317623ab-1f61-4d26-bde5-31d349ee78f7 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:48:15.011598 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317623ab-1f61-4d26-bde5-31d349ee78f7 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:48:15.058136 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6daf83-a560-40f2-b97a-211280d7ed1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:48:15.058881 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6daf83-a560-40f2-b97a-211280d7ed1c None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:48:15.059169 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6daf83-a560-40f2-b97a-211280d7ed1c None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:48:15.059459 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6daf83-a560-40f2-b97a-211280d7ed1c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:48:15.066620 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:48:15.072007 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6daf83-a560-40f2-b97a-211280d7ed1c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:48:15.073500 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6daf83-a560-40f2-b97a-211280d7ed1c None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:48:15.090474 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application [None req-bd6daf83-a560-40f2-b97a-211280d7ed1c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:48:15.090474 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:48:15.090474 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:48:15.090474 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:48:15.090474 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:48:15.090474 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:48:15.090474 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:48:15.090474 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:48:15.090474 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:48:15.090474 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:48:15.090474 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:48:15.090474 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:48:15.090474 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 10 11:48:15.090474 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 10 11:48:15.090474 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 10 11:48:15.090474 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 10 11:48:15.092591 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:48:15.092591 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:48:15.092591 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:48:15.092591 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:48:15.092591 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:48:15.092591 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:48:15.092591 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:48:15.092591 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application with Lock( Oct 10 11:48:15.092591 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:48:15.092591 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:48:15.092591 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:48:15.092591 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:48:15.092591 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:48:15.092591 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:48:15.092591 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:48:15.092591 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:48:15.092591 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 10 11:48:15.093699 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 10 11:48:15.093699 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:48:15.093699 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Oct 10 11:48:15.095079 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 537/1043] 10.222.0.46 () {56 vars in 1108 bytes} [Mon Oct 10 11:48:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:48:15.098789 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30571a04-d50e-46c7-929b-30e054eac669 None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:48:15.117658 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6daf83-a560-40f2-b97a-211280d7ed1c None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:48:15.233327 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:48:15.249587 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30571a04-d50e-46c7-929b-30e054eac669 None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:48:15.263619 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6daf83-a560-40f2-b97a-211280d7ed1c None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:48:15.298838 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942c547d-87bf-4491-8aef-875e0cb0207e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:48:15.299470 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942c547d-87bf-4491-8aef-875e0cb0207e None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:48:15.299756 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942c547d-87bf-4491-8aef-875e0cb0207e None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:48:15.300037 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942c547d-87bf-4491-8aef-875e0cb0207e None admin] PATH_INFO: `/v3/domains` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:48:15.300700 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942c547d-87bf-4491-8aef-875e0cb0207e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:48:15.301007 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942c547d-87bf-4491-8aef-875e0cb0207e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:48:15.301967 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6966feb7-7104-4d54-800f-7ca936221c86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:48:15.302426 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942c547d-87bf-4491-8aef-875e0cb0207e None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:48:15.302637 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6966feb7-7104-4d54-800f-7ca936221c86 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:48:15.302948 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6966feb7-7104-4d54-800f-7ca936221c86 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:48:15.303221 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6966feb7-7104-4d54-800f-7ca936221c86 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:48:15.315504 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 507/1044] 10.222.0.46 () {56 vars in 1109 bytes} [Mon Oct 10 11:48:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:48:15.315764 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6966feb7-7104-4d54-800f-7ca936221c86 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 10 11:48:15.316841 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7189bd39-db58-40f3-b915-342863381399 None None] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:48:15.317126 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7189bd39-db58-40f3-b915-342863381399 None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:48:15.317403 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7189bd39-db58-40f3-b915-342863381399 None None] PATH_INFO: `/` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:48:15.319160 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 508/1045] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 10 11:48:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:48:15.320778 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application [None req-6966feb7-7104-4d54-800f-7ca936221c86 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 10 11:48:15.320778 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:48:15.320778 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:48:15.320778 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:48:15.320778 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:48:15.320778 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:48:15.320778 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:48:15.320778 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:48:15.320778 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:48:15.320778 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:48:15.320778 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:48:15.320778 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:48:15.320778 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 10 11:48:15.320778 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 10 11:48:15.320778 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 10 11:48:15.320778 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 10 11:48:15.326059 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 10 11:48:15.326059 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 10 11:48:15.326059 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 10 11:48:15.326059 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 10 11:48:15.326059 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:48:15.326059 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:48:15.326059 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:48:15.326059 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:48:15.326059 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:48:15.326059 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:48:15.326059 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:48:15.326059 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application with Lock( Oct 10 11:48:15.326059 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:48:15.326059 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:48:15.326059 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:48:15.326059 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:48:15.327861 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:48:15.327861 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:48:15.327861 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:48:15.327861 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:48:15.327861 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 10 11:48:15.327861 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 10 11:48:15.327861 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 10 11:48:15.327861 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 10 11:48:15.327861 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 10 11:48:15.327861 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 10 11:48:15.327861 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 10 11:48:15.327861 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Oct 10 11:48:15.327861 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 538/1046] 10.222.0.46 () {56 vars in 1096 bytes} [Mon Oct 10 11:48:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 10 11:48:15.329191 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7189bd39-db58-40f3-b915-342863381399 None None] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:48:15.329323 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82aa3ea-9502-4c30-a313-4e4a46709a4b None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:48:15.329323 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82aa3ea-9502-4c30-a313-4e4a46709a4b None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:48:15.329323 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82aa3ea-9502-4c30-a313-4e4a46709a4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:48:15.347995 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-d82aa3ea-9502-4c30-a313-4e4a46709a4b None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:48:15.349323 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-d82aa3ea-9502-4c30-a313-4e4a46709a4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:48:15.402585 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:48:15.428081 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 539/1047] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 10 11:48:15 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 10 11:48:15.429013 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82aa3ea-9502-4c30-a313-4e4a46709a4b None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:48:15.475873 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7189bd39-db58-40f3-b915-342863381399 None None] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:48:15.536830 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a978d08-5f83-4587-9402-87baef399dbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:48:15.537606 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a978d08-5f83-4587-9402-87baef399dbb None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:48:15.538079 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a978d08-5f83-4587-9402-87baef399dbb None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:48:15.538536 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a978d08-5f83-4587-9402-87baef399dbb None admin] PATH_INFO: `/v3/projects/service` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:48:15.551385 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a978d08-5f83-4587-9402-87baef399dbb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:48:15.553645 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a978d08-5f83-4587-9402-87baef399dbb None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:48:15.574961 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application [None req-8a978d08-5f83-4587-9402-87baef399dbb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 10 11:48:15.574961 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:48:15.574961 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:48:15.574961 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:48:15.574961 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:48:15.574961 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:48:15.574961 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:48:15.574961 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:48:15.574961 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:48:15.574961 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:48:15.574961 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:48:15.574961 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:48:15.574961 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 10 11:48:15.574961 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 10 11:48:15.574961 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 10 11:48:15.575933 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 10 11:48:15.575933 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:48:15.575933 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:48:15.575933 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:48:15.575933 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:48:15.575933 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:48:15.575933 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:48:15.575933 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:48:15.575933 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application with Lock( Oct 10 11:48:15.575933 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:48:15.575933 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:48:15.575933 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:48:15.575933 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:48:15.575933 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:48:15.575933 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:48:15.575933 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:48:15.575933 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:48:15.576809 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 10 11:48:15.576809 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 10 11:48:15.576809 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 10 11:48:15.576809 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 10 11:48:15.576809 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 10 11:48:15.576809 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 10 11:48:15.576809 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 10 11:48:15.576809 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Oct 10 11:48:15.576809 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 509/1048] 10.222.0.46 () {56 vars in 1111 bytes} [Mon Oct 10 11:48:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:48:15.585148 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a978d08-5f83-4587-9402-87baef399dbb None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:48:15.594810 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82aa3ea-9502-4c30-a313-4e4a46709a4b None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:48:15.602077 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:48:15.645691 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba39a585-7842-45c5-ab34-db5b5f02c858 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:48:15.646322 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba39a585-7842-45c5-ab34-db5b5f02c858 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:48:15.646605 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba39a585-7842-45c5-ab34-db5b5f02c858 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:48:15.646918 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba39a585-7842-45c5-ab34-db5b5f02c858 None admin] PATH_INFO: `/v3/roles` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:48:15.647723 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba39a585-7842-45c5-ab34-db5b5f02c858 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:48:15.647905 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba39a585-7842-45c5-ab34-db5b5f02c858 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:48:15.649298 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba39a585-7842-45c5-ab34-db5b5f02c858 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:48:15.658489 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 540/1049] 10.222.0.46 () {56 vars in 1099 bytes} [Mon Oct 10 11:48:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:48:15.664391 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba39a585-7842-45c5-ab34-db5b5f02c858 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:48:15.717736 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:48:15.720709 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a978d08-5f83-4587-9402-87baef399dbb None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:48:15.762482 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e025748-f907-48da-af7a-ec6827706cf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:48:15.763221 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e025748-f907-48da-af7a-ec6827706cf1 None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:48:15.763675 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e025748-f907-48da-af7a-ec6827706cf1 None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:48:15.764120 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e025748-f907-48da-af7a-ec6827706cf1 None admin] PATH_INFO: `/v3/projects` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:48:15.765426 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e025748-f907-48da-af7a-ec6827706cf1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:48:15.765968 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e025748-f907-48da-af7a-ec6827706cf1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:48:15.768526 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e025748-f907-48da-af7a-ec6827706cf1 None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:48:15.783223 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 510/1050] 10.222.0.46 () {56 vars in 1148 bytes} [Mon Oct 10 11:48:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 11:48:15.784091 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e025748-f907-48da-af7a-ec6827706cf1 None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:48:15.786838 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba39a585-7842-45c5-ab34-db5b5f02c858 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:48:15.823472 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b207ad-4b4d-427e-ba3e-d539a4937888 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:48:15.824107 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b207ad-4b4d-427e-ba3e-d539a4937888 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:48:15.824388 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b207ad-4b4d-427e-ba3e-d539a4937888 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:48:15.824662 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b207ad-4b4d-427e-ba3e-d539a4937888 None admin] PATH_INFO: `/v3/users/004b90e236b54d0fb4b6c0aba5fa7b53` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:48:15.846991 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b207ad-4b4d-427e-ba3e-d539a4937888 None admin] RBAC: Authorizing `identity:get_user(user_id=004b90e236b54d0fb4b6c0aba5fa7b53)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:48:15.848349 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b207ad-4b4d-427e-ba3e-d539a4937888 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:48:15.850104 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 541/1051] 10.222.0.46 () {56 vars in 1177 bytes} [Mon Oct 10 11:48:15 2022] GET /identity/v3/users/004b90e236b54d0fb4b6c0aba5fa7b53 => generated 285 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 11:48:15.851019 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b207ad-4b4d-427e-ba3e-d539a4937888 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:48:15.877896 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:48:15.915409 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e025748-f907-48da-af7a-ec6827706cf1 None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:48:15.963099 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33dd3b1-9ef8-449f-b52c-7c69ae9b8d51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:48:15.963752 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33dd3b1-9ef8-449f-b52c-7c69ae9b8d51 None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:48:15.964041 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33dd3b1-9ef8-449f-b52c-7c69ae9b8d51 None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:48:15.964293 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33dd3b1-9ef8-449f-b52c-7c69ae9b8d51 None admin] PATH_INFO: `/v3/services/image` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:48:15.964913 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33dd3b1-9ef8-449f-b52c-7c69ae9b8d51 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:48:15.966335 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33dd3b1-9ef8-449f-b52c-7c69ae9b8d51 None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:48:15.977339 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application [None req-f33dd3b1-9ef8-449f-b52c-7c69ae9b8d51 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Oct 10 11:48:15.977339 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:48:15.977339 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 10 11:48:15.977339 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 10 11:48:15.977339 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 10 11:48:15.977339 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 10 11:48:15.977339 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 10 11:48:15.977339 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 10 11:48:15.977339 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 10 11:48:15.977339 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Oct 10 11:48:15.977339 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 10 11:48:15.977339 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Oct 10 11:48:15.977339 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:48:15.977339 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:48:15.977339 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:48:15.977339 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:48:15.977339 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:48:15.980030 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:48:15.980030 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:48:15.980030 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:48:15.980030 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:48:15.980030 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:48:15.980030 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:48:15.980030 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 10 11:48:15.980030 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 10 11:48:15.980030 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 10 11:48:15.980030 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 10 11:48:15.980030 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:48:15.980030 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:48:15.980030 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:48:15.980030 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:48:15.980030 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:48:15.980030 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:48:15.981685 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:48:15.981685 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application with Lock( Oct 10 11:48:15.981685 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:48:15.981685 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:48:15.981685 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:48:15.981685 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:48:15.981685 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:48:15.981685 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:48:15.981685 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:48:15.981685 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:48:15.981685 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 10 11:48:15.981685 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 10 11:48:15.981685 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 10 11:48:15.981685 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Oct 10 11:48:15.981685 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 511/1052] 10.222.0.46 () {56 vars in 1105 bytes} [Mon Oct 10 11:48:15 2022] GET /identity/v3/services/image => generated 86 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:48:15.983707 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33dd3b1-9ef8-449f-b52c-7c69ae9b8d51 None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:48:15.998055 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:48:16.001840 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b207ad-4b4d-427e-ba3e-d539a4937888 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:48:16.054773 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97cf04b4-34f8-467c-b802-696da4c646b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:48:16.055748 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cf04b4-34f8-467c-b802-696da4c646b1 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:48:16.056192 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cf04b4-34f8-467c-b802-696da4c646b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:48:16.057257 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cf04b4-34f8-467c-b802-696da4c646b1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:48:16.058490 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97cf04b4-34f8-467c-b802-696da4c646b1 None admin] RBAC: Adding query filter params (role.id=66c72919d8744554bc2e7c3dcc683e1f, scope.project.id=4ad3772f7b6a47c28b7ba37a9316a3c1, user.id=5f130bbb69e546ce99b4e14de81d7e54) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:48:16.059040 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97cf04b4-34f8-467c-b802-696da4c646b1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:48:16.061434 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97cf04b4-34f8-467c-b802-696da4c646b1 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:48:16.082705 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 542/1053] 10.222.0.46 () {56 vars in 1374 bytes} [Mon Oct 10 11:48:15 2022] GET /identity/v3/role_assignments?user.id=5f130bbb69e546ce99b4e14de81d7e54&scope.project.id=4ad3772f7b6a47c28b7ba37a9316a3c1&role.id=66c72919d8744554bc2e7c3dcc683e1f => generated 259 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:48:16.082705 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97cf04b4-34f8-467c-b802-696da4c646b1 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:48:16.149550 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33dd3b1-9ef8-449f-b52c-7c69ae9b8d51 None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:48:16.211088 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d94a31-d63d-4767-aa35-205af6e6d2a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:48:16.211929 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d94a31-d63d-4767-aa35-205af6e6d2a0 None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:48:16.212336 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d94a31-d63d-4767-aa35-205af6e6d2a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:48:16.212846 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d94a31-d63d-4767-aa35-205af6e6d2a0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:48:16.219474 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:48:16.286719 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97cf04b4-34f8-467c-b802-696da4c646b1 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:48:16.310948 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d94a31-d63d-4767-aa35-205af6e6d2a0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:48:16.312419 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d94a31-d63d-4767-aa35-205af6e6d2a0 None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:48:16.352055 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efdcd25-2651-47b2-9c87-865818f39674 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:48:16.353067 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efdcd25-2651-47b2-9c87-865818f39674 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:48:16.354110 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efdcd25-2651-47b2-9c87-865818f39674 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:48:16.354591 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efdcd25-2651-47b2-9c87-865818f39674 None admin] PATH_INFO: `/v3/services` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:48:16.355773 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9efdcd25-2651-47b2-9c87-865818f39674 None admin] RBAC: Adding query filter params (name=image) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:48:16.356223 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9efdcd25-2651-47b2-9c87-865818f39674 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:48:16.358938 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9efdcd25-2651-47b2-9c87-865818f39674 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:48:16.375240 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 543/1054] 10.222.0.46 () {56 vars in 1108 bytes} [Mon Oct 10 11:48:16 2022] GET /identity/v3/services?name=image => generated 122 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:48:16.378490 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efdcd25-2651-47b2-9c87-865818f39674 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:48:16.427880 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:48:16.483176 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 512/1055] 10.222.0.46 () {58 vars in 1282 bytes} [Mon Oct 10 11:48:15 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 503 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 10 11:48:16.494574 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d94a31-d63d-4767-aa35-205af6e6d2a0 None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:48:16.540317 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efdcd25-2651-47b2-9c87-865818f39674 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:48:16.553179 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:48:16.607333 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7438adc0-8003-45fa-9703-4733659c6266 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:48:16.608372 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7438adc0-8003-45fa-9703-4733659c6266 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:48:16.608847 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7438adc0-8003-45fa-9703-4733659c6266 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:48:16.609314 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7438adc0-8003-45fa-9703-4733659c6266 None admin] PATH_INFO: `/v3/services` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:48:16.610428 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7438adc0-8003-45fa-9703-4733659c6266 None admin] RBAC: Adding query filter params (type=image) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:48:16.610990 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7438adc0-8003-45fa-9703-4733659c6266 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:48:16.613575 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7438adc0-8003-45fa-9703-4733659c6266 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:48:16.625991 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 544/1056] 10.222.0.46 () {56 vars in 1108 bytes} [Mon Oct 10 11:48:16 2022] GET /identity/v3/services?type=image => generated 350 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 11:48:16.633996 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7438adc0-8003-45fa-9703-4733659c6266 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:48:16.674977 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d94a31-d63d-4767-aa35-205af6e6d2a0 None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:48:16.741496 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b10658-4d49-4934-86d3-928375ee6dda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:48:16.742454 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b10658-4d49-4934-86d3-928375ee6dda None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:48:16.743232 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b10658-4d49-4934-86d3-928375ee6dda None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:48:16.743662 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b10658-4d49-4934-86d3-928375ee6dda None admin] PATH_INFO: `/v3/projects/7d854c0679c94853b4c23aeb48cc8f9e` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:48:16.745822 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b10658-4d49-4934-86d3-928375ee6dda None admin] RBAC: Authorizing `identity:get_project(project_id=7d854c0679c94853b4c23aeb48cc8f9e)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:48:16.748847 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b10658-4d49-4934-86d3-928375ee6dda None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:48:16.751872 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 513/1057] 10.222.0.46 () {56 vars in 1186 bytes} [Mon Oct 10 11:48:16 2022] GET /identity/v3/projects/7d854c0679c94853b4c23aeb48cc8f9e => generated 311 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 11:48:16.757448 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:48:16.762393 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b10658-4d49-4934-86d3-928375ee6dda None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:48:16.768136 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7438adc0-8003-45fa-9703-4733659c6266 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:48:16.809126 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8968afd2-6878-4a14-864a-c94e939286a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:48:16.809752 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8968afd2-6878-4a14-864a-c94e939286a3 None admin] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:48:16.810038 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8968afd2-6878-4a14-864a-c94e939286a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:48:16.810342 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8968afd2-6878-4a14-864a-c94e939286a3 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:48:16.811035 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8968afd2-6878-4a14-864a-c94e939286a3 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:48:16.812412 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8968afd2-6878-4a14-864a-c94e939286a3 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:48:16.946705 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 545/1058] 10.222.0.46 () {60 vars in 1144 bytes} [Mon Oct 10 11:48:16 2022] POST /identity/v3/endpoints => generated 327 bytes in 303 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 11:48:16.946953 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b10658-4d49-4934-86d3-928375ee6dda None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:48:16.952783 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:48:17.035870 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1430cc9-a6b4-47c1-8ea9-35e6f850c400 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:48:17.036867 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1430cc9-a6b4-47c1-8ea9-35e6f850c400 None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:48:17.037372 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1430cc9-a6b4-47c1-8ea9-35e6f850c400 None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:48:17.037858 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1430cc9-a6b4-47c1-8ea9-35e6f850c400 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:48:17.039311 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1430cc9-a6b4-47c1-8ea9-35e6f850c400 None admin] RBAC: Adding query filter params (role.id=0d822a0674b6495fac623877a3b38025, scope.project.id=7d854c0679c94853b4c23aeb48cc8f9e, user.id=004b90e236b54d0fb4b6c0aba5fa7b53) {{(pid=81676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:48:17.039879 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1430cc9-a6b4-47c1-8ea9-35e6f850c400 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:48:17.043367 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1430cc9-a6b4-47c1-8ea9-35e6f850c400 None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:48:17.062707 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 514/1059] 10.222.0.46 () {56 vars in 1374 bytes} [Mon Oct 10 11:48:16 2022] GET /identity/v3/role_assignments?user.id=004b90e236b54d0fb4b6c0aba5fa7b53&scope.project.id=7d854c0679c94853b4c23aeb48cc8f9e&role.id=0d822a0674b6495fac623877a3b38025 => generated 259 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:48:18.977245 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add88361-8c92-4d9b-a62e-2ef5e7bc6557 None None] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:48:18.978016 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add88361-8c92-4d9b-a62e-2ef5e7bc6557 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:48:18.978865 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add88361-8c92-4d9b-a62e-2ef5e7bc6557 None None] PATH_INFO: `/` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:48:18.981561 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 546/1060] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 10 11:48:18 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:48:18.990120 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3aaa25-463f-4352-afcc-43f5a14a5dec None None] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:48:18.990989 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3aaa25-463f-4352-afcc-43f5a14a5dec None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:48:18.990989 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3aaa25-463f-4352-afcc-43f5a14a5dec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:48:19.022832 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.auth.core [None req-2f3aaa25-463f-4352-afcc-43f5a14a5dec None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:48:19.024890 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-2f3aaa25-463f-4352-afcc-43f5a14a5dec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:48:19.156614 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 515/1061] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 10 11:48:18 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 10 11:48:19.490769 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2917c893-3395-4ef7-ba7e-c73065c8fbf4 None None] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:48:19.491312 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2917c893-3395-4ef7-ba7e-c73065c8fbf4 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:48:19.491784 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2917c893-3395-4ef7-ba7e-c73065c8fbf4 None None] PATH_INFO: `/` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:48:19.495369 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 547/1062] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 10 11:48:19 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:48:19.501430 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec97f8af-db55-4841-a9b2-23defcdd5775 None None] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:48:19.501703 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec97f8af-db55-4841-a9b2-23defcdd5775 None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:48:19.502095 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec97f8af-db55-4841-a9b2-23defcdd5775 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:48:19.524441 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.auth.core [None req-ec97f8af-db55-4841-a9b2-23defcdd5775 None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:48:19.526224 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-ec97f8af-db55-4841-a9b2-23defcdd5775 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:48:19.609347 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 516/1063] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 10 11:48:19 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 10 11:48:19.660093 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2917c893-3395-4ef7-ba7e-c73065c8fbf4 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:48:19.787354 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2917c893-3395-4ef7-ba7e-c73065c8fbf4 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:48:19.826859 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0422c5a-3fdb-4640-b26e-f08c48841333 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:48:19.827507 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0422c5a-3fdb-4640-b26e-f08c48841333 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:48:19.827788 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0422c5a-3fdb-4640-b26e-f08c48841333 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:48:19.829159 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0422c5a-3fdb-4640-b26e-f08c48841333 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:48:19.840813 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0422c5a-3fdb-4640-b26e-f08c48841333 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 10 11:48:19.845740 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application [None req-a0422c5a-3fdb-4640-b26e-f08c48841333 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 10 11:48:19.845740 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:48:19.845740 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:48:19.845740 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:48:19.845740 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:48:19.845740 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:48:19.845740 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:48:19.845740 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:48:19.845740 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:48:19.845740 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:48:19.845740 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:48:19.845740 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:48:19.845740 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 10 11:48:19.845740 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 10 11:48:19.845740 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 10 11:48:19.845740 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 10 11:48:19.847738 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 10 11:48:19.847738 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 10 11:48:19.847738 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 10 11:48:19.847738 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 10 11:48:19.847738 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:48:19.847738 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:48:19.847738 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:48:19.847738 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:48:19.847738 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:48:19.847738 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:48:19.847738 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:48:19.847738 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application with Lock( Oct 10 11:48:19.847738 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:48:19.847738 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:48:19.847738 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:48:19.847738 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:48:19.849235 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:48:19.849235 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:48:19.849235 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:48:19.849235 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:48:19.849235 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 10 11:48:19.849235 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 10 11:48:19.849235 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 10 11:48:19.849235 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 10 11:48:19.849235 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 10 11:48:19.849235 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 10 11:48:19.849235 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 10 11:48:19.849235 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Oct 10 11:48:19.849235 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 548/1064] 10.222.0.46 () {56 vars in 1120 bytes} [Mon Oct 10 11:48:19 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:48:19.857508 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec97f8af-db55-4841-a9b2-23defcdd5775 None None] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:48:19.923718 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0422c5a-3fdb-4640-b26e-f08c48841333 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:48:19.944993 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:48:19.958276 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:48:20.119638 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec97f8af-db55-4841-a9b2-23defcdd5775 None None] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:48:20.161429 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0422c5a-3fdb-4640-b26e-f08c48841333 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:48:20.181226 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dcd088f-5703-4786-af48-64a19ebe1054 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:48:20.182264 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcd088f-5703-4786-af48-64a19ebe1054 None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:48:20.182848 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcd088f-5703-4786-af48-64a19ebe1054 None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:48:20.183442 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcd088f-5703-4786-af48-64a19ebe1054 None admin] PATH_INFO: `/v3/roles` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:48:20.184687 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dcd088f-5703-4786-af48-64a19ebe1054 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:48:20.185232 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dcd088f-5703-4786-af48-64a19ebe1054 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:48:20.188193 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dcd088f-5703-4786-af48-64a19ebe1054 None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:48:20.206753 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 517/1065] 10.222.0.46 () {56 vars in 1115 bytes} [Mon Oct 10 11:48:19 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 348 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:48:20.210051 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dcd088f-5703-4786-af48-64a19ebe1054 None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:48:20.222262 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdf3242-7a5c-45f5-896f-84259b5e797a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:48:20.223272 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdf3242-7a5c-45f5-896f-84259b5e797a None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:48:20.223740 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdf3242-7a5c-45f5-896f-84259b5e797a None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:48:20.224196 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdf3242-7a5c-45f5-896f-84259b5e797a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:48:20.325916 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:48:20.332368 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fdf3242-7a5c-45f5-896f-84259b5e797a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:48:20.341913 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fdf3242-7a5c-45f5-896f-84259b5e797a None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:48:20.342127 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dcd088f-5703-4786-af48-64a19ebe1054 None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:48:20.354295 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:48:20.392766 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac69ca4-7fab-4634-8693-73d9d1159940 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:48:20.393642 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac69ca4-7fab-4634-8693-73d9d1159940 None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:48:20.394093 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac69ca4-7fab-4634-8693-73d9d1159940 None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:48:20.394535 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac69ca4-7fab-4634-8693-73d9d1159940 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:48:20.411702 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac69ca4-7fab-4634-8693-73d9d1159940 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:48:20.414058 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac69ca4-7fab-4634-8693-73d9d1159940 None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:48:20.440343 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application [None req-8ac69ca4-7fab-4634-8693-73d9d1159940 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:48:20.440343 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:48:20.440343 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:48:20.440343 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:48:20.440343 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:48:20.440343 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:48:20.440343 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:48:20.440343 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:48:20.440343 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:48:20.440343 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:48:20.440343 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:48:20.440343 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:48:20.440343 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 10 11:48:20.440343 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 10 11:48:20.440343 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 10 11:48:20.440343 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 10 11:48:20.442234 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:48:20.442234 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:48:20.442234 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:48:20.442234 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:48:20.442234 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:48:20.442234 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:48:20.442234 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:48:20.442234 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application with Lock( Oct 10 11:48:20.442234 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:48:20.442234 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:48:20.442234 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:48:20.442234 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:48:20.442234 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:48:20.442234 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:48:20.442234 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:48:20.442234 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:48:20.442234 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 10 11:48:20.443353 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 10 11:48:20.443353 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:48:20.443353 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Oct 10 11:48:20.443353 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 518/1066] 10.222.0.46 () {56 vars in 1108 bytes} [Mon Oct 10 11:48:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:48:20.444689 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd47f470-1f35-4ca2-a6b0-3570684ac086 None None] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:48:20.445165 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd47f470-1f35-4ca2-a6b0-3570684ac086 None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:48:20.445643 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd47f470-1f35-4ca2-a6b0-3570684ac086 None None] PATH_INFO: `/` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:48:20.449292 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 519/1067] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 10 11:48:20 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:48:20.452169 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd47f470-1f35-4ca2-a6b0-3570684ac086 None None] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:48:20.501620 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 549/1068] 10.222.0.46 () {58 vars in 1282 bytes} [Mon Oct 10 11:48:19 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 579 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 10 11:48:20.503375 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee4ae49-7ccf-4428-8868-3e6737daa6ff None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:48:20.503915 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee4ae49-7ccf-4428-8868-3e6737daa6ff None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:48:20.504352 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee4ae49-7ccf-4428-8868-3e6737daa6ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:48:20.531839 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-fee4ae49-7ccf-4428-8868-3e6737daa6ff None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:48:20.533742 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-fee4ae49-7ccf-4428-8868-3e6737daa6ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:48:20.609379 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd47f470-1f35-4ca2-a6b0-3570684ac086 None None] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:48:20.630567 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 550/1069] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 10 11:48:20 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 10 11:48:20.632964 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee4ae49-7ccf-4428-8868-3e6737daa6ff None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:48:20.670106 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000b0ddb-4065-46de-bfe4-8d4cf8f3eff9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:48:20.671079 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000b0ddb-4065-46de-bfe4-8d4cf8f3eff9 None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:48:20.671557 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000b0ddb-4065-46de-bfe4-8d4cf8f3eff9 None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:48:20.672077 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000b0ddb-4065-46de-bfe4-8d4cf8f3eff9 None admin] PATH_INFO: `/v3/domains` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:48:20.673302 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000b0ddb-4065-46de-bfe4-8d4cf8f3eff9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:48:20.673838 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000b0ddb-4065-46de-bfe4-8d4cf8f3eff9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:48:20.676605 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000b0ddb-4065-46de-bfe4-8d4cf8f3eff9 None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:48:20.685875 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:48:20.689686 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:48:20.700053 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 520/1070] 10.222.0.46 () {56 vars in 1109 bytes} [Mon Oct 10 11:48:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 11:48:20.709282 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000b0ddb-4065-46de-bfe4-8d4cf8f3eff9 None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:48:20.786907 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee4ae49-7ccf-4428-8868-3e6737daa6ff None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:48:20.849346 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7676ba3c-7ac3-42b9-bea8-5e9138a9908c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:48:20.850992 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7676ba3c-7ac3-42b9-bea8-5e9138a9908c None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:48:20.850992 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7676ba3c-7ac3-42b9-bea8-5e9138a9908c None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:48:20.851460 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7676ba3c-7ac3-42b9-bea8-5e9138a9908c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:48:20.870257 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7676ba3c-7ac3-42b9-bea8-5e9138a9908c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:48:20.872332 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7676ba3c-7ac3-42b9-bea8-5e9138a9908c None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:48:20.892802 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000b0ddb-4065-46de-bfe4-8d4cf8f3eff9 None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:48:20.904210 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application [None req-7676ba3c-7ac3-42b9-bea8-5e9138a9908c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:48:20.904210 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:48:20.904210 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:48:20.904210 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:48:20.904210 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:48:20.904210 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:48:20.904210 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:48:20.904210 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:48:20.904210 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:48:20.904210 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:48:20.904210 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:48:20.904210 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:48:20.904210 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 10 11:48:20.904210 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 10 11:48:20.904210 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 10 11:48:20.904210 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 10 11:48:20.906054 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:48:20.906054 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:48:20.906054 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:48:20.906054 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:48:20.906054 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:48:20.906054 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:48:20.906054 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:48:20.906054 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application with Lock( Oct 10 11:48:20.906054 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:48:20.906054 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:48:20.906054 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:48:20.906054 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:48:20.906054 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:48:20.906054 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:48:20.906054 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:48:20.906054 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:48:20.906054 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 10 11:48:20.907923 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 10 11:48:20.907923 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:48:20.907923 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Oct 10 11:48:20.907923 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 551/1071] 10.222.0.46 () {56 vars in 1108 bytes} [Mon Oct 10 11:48:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:48:20.914049 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7676ba3c-7ac3-42b9-bea8-5e9138a9908c None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:48:20.962103 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5162198-2ada-4dff-bfec-faabc146367a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:48:20.963659 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5162198-2ada-4dff-bfec-faabc146367a None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:48:20.964165 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5162198-2ada-4dff-bfec-faabc146367a None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:48:20.964707 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5162198-2ada-4dff-bfec-faabc146367a None admin] PATH_INFO: `/v3/users/nova` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:48:20.978913 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5162198-2ada-4dff-bfec-faabc146367a None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:48:20.980381 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5162198-2ada-4dff-bfec-faabc146367a None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:48:20.993579 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application [None req-b5162198-2ada-4dff-bfec-faabc146367a None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 10 11:48:20.993579 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:48:20.993579 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:48:20.993579 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:48:20.993579 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:48:20.993579 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:48:20.993579 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:48:20.993579 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:48:20.993579 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:48:20.993579 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:48:20.993579 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:48:20.993579 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:48:20.993579 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 10 11:48:20.993579 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 10 11:48:20.993579 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 10 11:48:20.993579 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 10 11:48:20.995582 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:48:20.995582 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:48:20.995582 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 10 11:48:20.995582 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 10 11:48:20.995582 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 10 11:48:20.995582 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 10 11:48:20.995582 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:48:20.995582 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:48:20.995582 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:48:20.995582 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:48:20.995582 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:48:20.995582 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application with Lock( Oct 10 11:48:20.995582 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:48:20.995582 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:48:20.995582 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:48:20.995582 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:48:20.995582 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:48:20.995582 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:48:20.996891 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:48:20.996891 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:48:20.996891 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 10 11:48:20.996891 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 10 11:48:20.996891 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 10 11:48:20.996891 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 10 11:48:20.996891 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 10 11:48:20.996891 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 10 11:48:20.996891 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 10 11:48:20.996891 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Oct 10 11:48:20.996891 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 521/1072] 10.222.0.46 () {56 vars in 1093 bytes} [Mon Oct 10 11:48:20 2022] GET /identity/v3/users/nova => generated 82 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 10 11:48:20.996891 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5162198-2ada-4dff-bfec-faabc146367a None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:48:21.038064 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:48:21.053279 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7676ba3c-7ac3-42b9-bea8-5e9138a9908c None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:48:21.082754 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:48:21.107104 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05b0099-84ac-40d2-8f58-ad4cba94d0c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:48:21.107730 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05b0099-84ac-40d2-8f58-ad4cba94d0c4 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:48:21.108145 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05b0099-84ac-40d2-8f58-ad4cba94d0c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:48:21.108346 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05b0099-84ac-40d2-8f58-ad4cba94d0c4 None admin] PATH_INFO: `/v3/domains` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:48:21.108984 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05b0099-84ac-40d2-8f58-ad4cba94d0c4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:48:21.109285 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05b0099-84ac-40d2-8f58-ad4cba94d0c4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:48:21.110719 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05b0099-84ac-40d2-8f58-ad4cba94d0c4 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:48:21.126029 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 552/1073] 10.222.0.46 () {56 vars in 1109 bytes} [Mon Oct 10 11:48:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:48:21.127548 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05b0099-84ac-40d2-8f58-ad4cba94d0c4 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:48:21.136096 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5162198-2ada-4dff-bfec-faabc146367a None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:48:21.187558 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbe5a01-c464-434d-919c-7f7bf0204291 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:48:21.188639 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbe5a01-c464-434d-919c-7f7bf0204291 None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:48:21.189124 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbe5a01-c464-434d-919c-7f7bf0204291 None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:48:21.189596 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbe5a01-c464-434d-919c-7f7bf0204291 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:48:21.210825 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fbe5a01-c464-434d-919c-7f7bf0204291 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 10 11:48:21.220217 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application [None req-5fbe5a01-c464-434d-919c-7f7bf0204291 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 10 11:48:21.220217 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:48:21.220217 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:48:21.220217 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:48:21.220217 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:48:21.220217 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:48:21.220217 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:48:21.220217 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:48:21.220217 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:48:21.220217 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:48:21.220217 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:48:21.220217 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:48:21.220217 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 10 11:48:21.220217 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 10 11:48:21.220217 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 10 11:48:21.220217 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 10 11:48:21.221220 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 10 11:48:21.221220 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 10 11:48:21.221220 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 10 11:48:21.221220 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 10 11:48:21.221220 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:48:21.221220 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:48:21.221220 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:48:21.221220 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:48:21.221220 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:48:21.221220 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:48:21.221220 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:48:21.221220 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application with Lock( Oct 10 11:48:21.221220 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:48:21.221220 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:48:21.221220 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:48:21.221220 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:48:21.222135 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:48:21.222135 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:48:21.222135 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:48:21.222135 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:48:21.222135 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 10 11:48:21.222135 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 10 11:48:21.222135 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 10 11:48:21.222135 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 10 11:48:21.222135 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 10 11:48:21.222135 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 10 11:48:21.222135 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 10 11:48:21.222135 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Oct 10 11:48:21.222135 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 522/1074] 10.222.0.46 () {56 vars in 1096 bytes} [Mon Oct 10 11:48:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:48:21.229170 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbe5a01-c464-434d-919c-7f7bf0204291 None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:48:21.328897 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05b0099-84ac-40d2-8f58-ad4cba94d0c4 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:48:21.386116 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbe5a01-c464-434d-919c-7f7bf0204291 None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:48:21.387917 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c409db-38eb-48a1-9f53-4b5e28852f8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:48:21.389213 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c409db-38eb-48a1-9f53-4b5e28852f8e None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:48:21.389213 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c409db-38eb-48a1-9f53-4b5e28852f8e None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:48:21.389213 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c409db-38eb-48a1-9f53-4b5e28852f8e None admin] PATH_INFO: `/v3/users` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:48:21.391339 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c409db-38eb-48a1-9f53-4b5e28852f8e None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:48:21.391339 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c409db-38eb-48a1-9f53-4b5e28852f8e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:48:21.394331 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c409db-38eb-48a1-9f53-4b5e28852f8e None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:48:21.417896 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 553/1075] 10.222.0.46 () {56 vars in 1133 bytes} [Mon Oct 10 11:48:21 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:48:21.418223 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:48:21.427719 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c409db-38eb-48a1-9f53-4b5e28852f8e None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:48:21.440528 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8edd5b5e-dcd7-4025-b907-7f7ef0451740 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:48:21.440811 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edd5b5e-dcd7-4025-b907-7f7ef0451740 None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:48:21.440996 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edd5b5e-dcd7-4025-b907-7f7ef0451740 None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:48:21.441187 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edd5b5e-dcd7-4025-b907-7f7ef0451740 None admin] PATH_INFO: `/v3/roles` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:48:21.442054 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8edd5b5e-dcd7-4025-b907-7f7ef0451740 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:48:21.442251 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8edd5b5e-dcd7-4025-b907-7f7ef0451740 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:48:21.443620 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8edd5b5e-dcd7-4025-b907-7f7ef0451740 None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:48:21.444992 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:48:21.463508 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 523/1076] 10.222.0.46 () {56 vars in 1099 bytes} [Mon Oct 10 11:48:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 11:48:21.470643 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8edd5b5e-dcd7-4025-b907-7f7ef0451740 None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:48:21.603835 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c409db-38eb-48a1-9f53-4b5e28852f8e None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:48:21.610491 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8edd5b5e-dcd7-4025-b907-7f7ef0451740 None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:48:21.648902 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74d2854-cdc6-4918-a3ae-5ca714076fd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:48:21.649980 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74d2854-cdc6-4918-a3ae-5ca714076fd3 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:48:21.650490 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74d2854-cdc6-4918-a3ae-5ca714076fd3 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:48:21.650961 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74d2854-cdc6-4918-a3ae-5ca714076fd3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:48:21.653095 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c43b005-198c-4eaf-870c-d223192f1c7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:48:21.654065 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c43b005-198c-4eaf-870c-d223192f1c7b None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:48:21.654513 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c43b005-198c-4eaf-870c-d223192f1c7b None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:48:21.655000 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c43b005-198c-4eaf-870c-d223192f1c7b None admin] PATH_INFO: `/v3/users/004b90e236b54d0fb4b6c0aba5fa7b53` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:48:21.656765 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c43b005-198c-4eaf-870c-d223192f1c7b None admin] RBAC: Authorizing `identity:get_user(user_id=004b90e236b54d0fb4b6c0aba5fa7b53)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:48:21.659351 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c43b005-198c-4eaf-870c-d223192f1c7b None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:48:21.662238 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 524/1077] 10.222.0.46 () {56 vars in 1177 bytes} [Mon Oct 10 11:48:21 2022] GET /identity/v3/users/004b90e236b54d0fb4b6c0aba5fa7b53 => generated 285 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 11:48:21.668837 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74d2854-cdc6-4918-a3ae-5ca714076fd3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:48:21.671600 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c43b005-198c-4eaf-870c-d223192f1c7b None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:48:21.671755 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74d2854-cdc6-4918-a3ae-5ca714076fd3 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:48:21.693884 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application [None req-e74d2854-cdc6-4918-a3ae-5ca714076fd3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:48:21.693884 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:48:21.693884 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:48:21.693884 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:48:21.693884 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:48:21.693884 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:48:21.693884 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:48:21.693884 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:48:21.693884 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:48:21.693884 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:48:21.693884 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:48:21.693884 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:48:21.693884 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 10 11:48:21.693884 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 10 11:48:21.693884 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 10 11:48:21.693884 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 10 11:48:21.704171 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:48:21.704171 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:48:21.704171 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:48:21.704171 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:48:21.704171 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:48:21.704171 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:48:21.704171 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:48:21.704171 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application with Lock( Oct 10 11:48:21.704171 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:48:21.704171 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:48:21.704171 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:48:21.704171 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:48:21.704171 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:48:21.704171 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:48:21.704171 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:48:21.704171 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:48:21.704171 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 10 11:48:21.706214 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 10 11:48:21.706214 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:48:21.706214 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Oct 10 11:48:21.706214 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 554/1078] 10.222.0.46 () {56 vars in 1108 bytes} [Mon Oct 10 11:48:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 10 11:48:21.706214 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74d2854-cdc6-4918-a3ae-5ca714076fd3 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:48:21.744176 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:48:21.793712 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:48:21.826851 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c43b005-198c-4eaf-870c-d223192f1c7b None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:48:21.840826 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74d2854-cdc6-4918-a3ae-5ca714076fd3 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:48:21.869695 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9882d283-c095-4c80-ae5b-d74b498b77a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:48:21.870349 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9882d283-c095-4c80-ae5b-d74b498b77a0 None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:48:21.870653 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9882d283-c095-4c80-ae5b-d74b498b77a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:48:21.870931 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9882d283-c095-4c80-ae5b-d74b498b77a0 None admin] PATH_INFO: `/v3/projects/7d854c0679c94853b4c23aeb48cc8f9e` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:48:21.872123 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9882d283-c095-4c80-ae5b-d74b498b77a0 None admin] RBAC: Authorizing `identity:get_project(project_id=7d854c0679c94853b4c23aeb48cc8f9e)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:48:21.873568 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9882d283-c095-4c80-ae5b-d74b498b77a0 None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:48:21.875351 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 525/1079] 10.222.0.46 () {56 vars in 1186 bytes} [Mon Oct 10 11:48:21 2022] GET /identity/v3/projects/7d854c0679c94853b4c23aeb48cc8f9e => generated 311 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 11:48:21.881567 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9882d283-c095-4c80-ae5b-d74b498b77a0 None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:48:21.891394 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36aee77-0b40-4d2b-8913-07ca8af3e344 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:48:21.892012 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36aee77-0b40-4d2b-8913-07ca8af3e344 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:48:21.892317 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36aee77-0b40-4d2b-8913-07ca8af3e344 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:48:21.892592 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36aee77-0b40-4d2b-8913-07ca8af3e344 None admin] PATH_INFO: `/v3/domains` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:48:21.893242 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36aee77-0b40-4d2b-8913-07ca8af3e344 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:48:21.893548 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36aee77-0b40-4d2b-8913-07ca8af3e344 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:48:21.894948 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36aee77-0b40-4d2b-8913-07ca8af3e344 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:48:21.908179 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 555/1080] 10.222.0.46 () {56 vars in 1109 bytes} [Mon Oct 10 11:48:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 11:48:21.913991 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36aee77-0b40-4d2b-8913-07ca8af3e344 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:48:22.016364 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36aee77-0b40-4d2b-8913-07ca8af3e344 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:48:22.037959 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:48:22.053511 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe4fc82-e3aa-4e64-8dfe-68a5c20c6456 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:48:22.057031 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe4fc82-e3aa-4e64-8dfe-68a5c20c6456 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:48:22.057031 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe4fc82-e3aa-4e64-8dfe-68a5c20c6456 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:48:22.057031 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe4fc82-e3aa-4e64-8dfe-68a5c20c6456 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:48:22.059858 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9882d283-c095-4c80-ae5b-d74b498b77a0 None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:48:22.061868 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe4fc82-e3aa-4e64-8dfe-68a5c20c6456 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:48:22.063442 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe4fc82-e3aa-4e64-8dfe-68a5c20c6456 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:48:22.077734 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application [None req-fbe4fc82-e3aa-4e64-8dfe-68a5c20c6456 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 10 11:48:22.077734 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:48:22.077734 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:48:22.077734 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:48:22.077734 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:48:22.077734 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:48:22.077734 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:48:22.077734 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:48:22.077734 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:48:22.077734 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:48:22.077734 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:48:22.077734 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:48:22.077734 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 10 11:48:22.077734 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 10 11:48:22.077734 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 10 11:48:22.079520 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 10 11:48:22.079520 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:48:22.079520 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:48:22.079520 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:48:22.079520 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:48:22.079520 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:48:22.079520 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:48:22.079520 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:48:22.079520 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application with Lock( Oct 10 11:48:22.079520 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:48:22.079520 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:48:22.079520 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:48:22.079520 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:48:22.079520 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:48:22.079520 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:48:22.079520 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:48:22.079520 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:48:22.086225 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 10 11:48:22.086225 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 10 11:48:22.086225 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 10 11:48:22.086225 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 10 11:48:22.086225 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 10 11:48:22.086225 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 10 11:48:22.086225 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 10 11:48:22.086225 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Oct 10 11:48:22.086225 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 556/1081] 10.222.0.46 () {56 vars in 1111 bytes} [Mon Oct 10 11:48:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 10 11:48:22.086806 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:48:22.087401 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe4fc82-e3aa-4e64-8dfe-68a5c20c6456 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:48:22.108823 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025ba602-9a98-4282-992f-d19e6a6c4dcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:48:22.110470 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025ba602-9a98-4282-992f-d19e6a6c4dcb None admin] REQUEST_METHOD: `PUT` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:48:22.110470 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025ba602-9a98-4282-992f-d19e6a6c4dcb None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:48:22.110470 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025ba602-9a98-4282-992f-d19e6a6c4dcb None admin] PATH_INFO: `/v3/projects/7d854c0679c94853b4c23aeb48cc8f9e/users/004b90e236b54d0fb4b6c0aba5fa7b53/roles/0d822a0674b6495fac623877a3b38025` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:48:22.112868 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025ba602-9a98-4282-992f-d19e6a6c4dcb None admin] RBAC: Authorizing `identity:create_grant(project_id=7d854c0679c94853b4c23aeb48cc8f9e, user_id=004b90e236b54d0fb4b6c0aba5fa7b53, role_id=0d822a0674b6495fac623877a3b38025)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:48:22.114725 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025ba602-9a98-4282-992f-d19e6a6c4dcb None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:48:22.200956 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe4fc82-e3aa-4e64-8dfe-68a5c20c6456 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:48:22.217368 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 526/1082] 10.222.0.46 () {58 vars in 1439 bytes} [Mon Oct 10 11:48:21 2022] PUT /identity/v3/projects/7d854c0679c94853b4c23aeb48cc8f9e/users/004b90e236b54d0fb4b6c0aba5fa7b53/roles/0d822a0674b6495fac623877a3b38025 => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 10 11:48:22.255191 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849e960a-9d6c-4875-befc-901d63739a79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:48:22.256095 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849e960a-9d6c-4875-befc-901d63739a79 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:48:22.256501 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849e960a-9d6c-4875-befc-901d63739a79 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:48:22.256915 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849e960a-9d6c-4875-befc-901d63739a79 None admin] PATH_INFO: `/v3/projects` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:48:22.257998 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849e960a-9d6c-4875-befc-901d63739a79 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:48:22.258475 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849e960a-9d6c-4875-befc-901d63739a79 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:48:22.261465 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849e960a-9d6c-4875-befc-901d63739a79 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:48:22.283860 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 557/1083] 10.222.0.46 () {56 vars in 1148 bytes} [Mon Oct 10 11:48:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:48:22.290530 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025ba602-9a98-4282-992f-d19e6a6c4dcb None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:48:22.417680 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025ba602-9a98-4282-992f-d19e6a6c4dcb None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:48:22.464418 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0189a01-a65c-43b7-9487-cc78e603b283 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:48:22.465310 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0189a01-a65c-43b7-9487-cc78e603b283 None admin] REQUEST_METHOD: `PUT` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:48:22.465725 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0189a01-a65c-43b7-9487-cc78e603b283 None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:48:22.466136 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0189a01-a65c-43b7-9487-cc78e603b283 None admin] PATH_INFO: `/v3/projects/4ad3772f7b6a47c28b7ba37a9316a3c1/users/5f130bbb69e546ce99b4e14de81d7e54/roles/66c72919d8744554bc2e7c3dcc683e1f` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:48:22.468737 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0189a01-a65c-43b7-9487-cc78e603b283 None admin] RBAC: Authorizing `identity:create_grant(project_id=4ad3772f7b6a47c28b7ba37a9316a3c1, user_id=5f130bbb69e546ce99b4e14de81d7e54, role_id=66c72919d8744554bc2e7c3dcc683e1f)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:48:22.470435 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0189a01-a65c-43b7-9487-cc78e603b283 None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:48:22.595741 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 527/1084] 10.222.0.46 () {58 vars in 1439 bytes} [Mon Oct 10 11:48:22 2022] PUT /identity/v3/projects/4ad3772f7b6a47c28b7ba37a9316a3c1/users/5f130bbb69e546ce99b4e14de81d7e54/roles/66c72919d8744554bc2e7c3dcc683e1f => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 11:48:24.495240 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8cf48a-67d8-4210-b25b-92c12b897053 None None] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:48:24.496089 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8cf48a-67d8-4210-b25b-92c12b897053 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:48:24.496089 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8cf48a-67d8-4210-b25b-92c12b897053 None None] PATH_INFO: `/` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:48:24.497215 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 558/1085] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 10 11:48:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:48:24.503706 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b9ac69-cb4a-487b-92ba-bebf9450b188 None None] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:48:24.504279 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b9ac69-cb4a-487b-92ba-bebf9450b188 None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:48:24.504279 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b9ac69-cb4a-487b-92ba-bebf9450b188 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:48:24.525139 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.auth.core [None req-76b9ac69-cb4a-487b-92ba-bebf9450b188 None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:48:24.526445 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-76b9ac69-cb4a-487b-92ba-bebf9450b188 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:48:24.563133 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:48:24.606176 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 528/1086] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 10 11:48:24 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 10 11:48:24.885450 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8cf48a-67d8-4210-b25b-92c12b897053 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:48:24.906806 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55f5cbb-59c4-4611-a031-9666860ff17d None None] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:48:24.907114 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55f5cbb-59c4-4611-a031-9666860ff17d None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:48:24.907349 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55f5cbb-59c4-4611-a031-9666860ff17d None None] PATH_INFO: `/` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:48:24.908760 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 529/1087] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 10 11:48:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 10 11:48:24.915028 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cd38f4-f424-4a0c-b4d2-87e219591386 None None] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:48:24.915350 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cd38f4-f424-4a0c-b4d2-87e219591386 None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:48:24.916105 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cd38f4-f424-4a0c-b4d2-87e219591386 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:48:24.936846 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.auth.core [None req-a0cd38f4-f424-4a0c-b4d2-87e219591386 None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:48:24.938108 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-a0cd38f4-f424-4a0c-b4d2-87e219591386 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:48:24.948965 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:48:25.031394 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8cf48a-67d8-4210-b25b-92c12b897053 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:48:25.037329 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 530/1088] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 10 11:48:24 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 10 11:48:25.072408 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7db1f5-53c6-431f-9720-f75ea6a7a6a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:48:25.073046 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7db1f5-53c6-431f-9720-f75ea6a7a6a3 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:48:25.073326 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7db1f5-53c6-431f-9720-f75ea6a7a6a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:48:25.073594 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7db1f5-53c6-431f-9720-f75ea6a7a6a3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:48:25.085276 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7db1f5-53c6-431f-9720-f75ea6a7a6a3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 10 11:48:25.090886 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application [None req-0f7db1f5-53c6-431f-9720-f75ea6a7a6a3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 10 11:48:25.090886 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:48:25.090886 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:48:25.090886 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:48:25.090886 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:48:25.090886 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:48:25.090886 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:48:25.090886 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:48:25.090886 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:48:25.090886 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:48:25.090886 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:48:25.090886 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:48:25.090886 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 10 11:48:25.090886 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 10 11:48:25.090886 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 10 11:48:25.090886 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 10 11:48:25.092458 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 10 11:48:25.092458 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 10 11:48:25.092458 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 10 11:48:25.092458 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 10 11:48:25.092458 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:48:25.092458 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:48:25.092458 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:48:25.092458 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:48:25.092458 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:48:25.092458 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:48:25.092458 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:48:25.092458 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application with Lock( Oct 10 11:48:25.092458 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:48:25.092458 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:48:25.092458 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:48:25.092458 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:48:25.093644 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:48:25.093644 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:48:25.093644 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:48:25.093644 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:48:25.093644 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 10 11:48:25.093644 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 10 11:48:25.093644 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 10 11:48:25.093644 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 10 11:48:25.093644 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 10 11:48:25.093644 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 10 11:48:25.093644 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 10 11:48:25.093644 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Oct 10 11:48:25.093644 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 559/1089] 10.222.0.46 () {56 vars in 1096 bytes} [Mon Oct 10 11:48:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:48:25.099090 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0cd38f4-f424-4a0c-b4d2-87e219591386 None None] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:48:25.215181 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:48:25.221557 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0cd38f4-f424-4a0c-b4d2-87e219591386 None None] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:48:25.265086 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d41500f-39ec-4058-8a5e-eef46b3fa105 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:48:25.265712 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d41500f-39ec-4058-8a5e-eef46b3fa105 None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:48:25.265991 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d41500f-39ec-4058-8a5e-eef46b3fa105 None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:48:25.266273 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d41500f-39ec-4058-8a5e-eef46b3fa105 None admin] PATH_INFO: `/v3/roles` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:48:25.266991 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d41500f-39ec-4058-8a5e-eef46b3fa105 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:48:25.267295 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d41500f-39ec-4058-8a5e-eef46b3fa105 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:48:25.268710 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d41500f-39ec-4058-8a5e-eef46b3fa105 None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:48:25.278745 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 531/1090] 10.222.0.46 () {56 vars in 1099 bytes} [Mon Oct 10 11:48:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:48:25.284954 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7db1f5-53c6-431f-9720-f75ea6a7a6a3 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:48:25.311672 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d41500f-39ec-4058-8a5e-eef46b3fa105 None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:48:25.409560 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:48:25.413219 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7db1f5-53c6-431f-9720-f75ea6a7a6a3 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:48:25.431705 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d41500f-39ec-4058-8a5e-eef46b3fa105 None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:48:25.456025 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2b7d79-e8ea-4a67-9e52-81578707dc99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:48:25.456664 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2b7d79-e8ea-4a67-9e52-81578707dc99 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:48:25.456943 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2b7d79-e8ea-4a67-9e52-81578707dc99 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:48:25.457211 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2b7d79-e8ea-4a67-9e52-81578707dc99 None admin] PATH_INFO: `/v3/users/004b90e236b54d0fb4b6c0aba5fa7b53` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:48:25.458375 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2b7d79-e8ea-4a67-9e52-81578707dc99 None admin] RBAC: Authorizing `identity:get_user(user_id=004b90e236b54d0fb4b6c0aba5fa7b53)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:48:25.459854 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2b7d79-e8ea-4a67-9e52-81578707dc99 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:48:25.466521 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 560/1091] 10.222.0.46 () {56 vars in 1177 bytes} [Mon Oct 10 11:48:25 2022] GET /identity/v3/users/004b90e236b54d0fb4b6c0aba5fa7b53 => generated 285 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:48:25.467375 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2b7d79-e8ea-4a67-9e52-81578707dc99 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:48:25.473018 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb918aa4-ee7c-4623-9dfd-d54a526de982 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:48:25.474722 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb918aa4-ee7c-4623-9dfd-d54a526de982 None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:48:25.475124 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb918aa4-ee7c-4623-9dfd-d54a526de982 None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:48:25.475507 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb918aa4-ee7c-4623-9dfd-d54a526de982 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:48:25.490411 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb918aa4-ee7c-4623-9dfd-d54a526de982 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 10 11:48:25.496175 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application [None req-cb918aa4-ee7c-4623-9dfd-d54a526de982 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 10 11:48:25.496175 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:48:25.496175 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:48:25.496175 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:48:25.496175 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:48:25.496175 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:48:25.496175 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:48:25.496175 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:48:25.496175 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:48:25.496175 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:48:25.496175 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:48:25.496175 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:48:25.496175 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 10 11:48:25.496175 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 10 11:48:25.496175 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 10 11:48:25.496175 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 10 11:48:25.497305 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 10 11:48:25.497305 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 10 11:48:25.497305 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 10 11:48:25.497305 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 10 11:48:25.497305 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:48:25.497305 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:48:25.497305 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:48:25.497305 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:48:25.497305 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:48:25.497305 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:48:25.497305 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:48:25.497305 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application with Lock( Oct 10 11:48:25.497305 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:48:25.497305 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:48:25.497305 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:48:25.497305 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:48:25.498771 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:48:25.498771 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:48:25.498771 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:48:25.498771 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:48:25.498771 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 10 11:48:25.498771 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 10 11:48:25.498771 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 10 11:48:25.498771 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 10 11:48:25.498771 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 10 11:48:25.498771 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 10 11:48:25.498771 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 10 11:48:25.498771 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Oct 10 11:48:25.498771 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 532/1092] 10.222.0.46 () {56 vars in 1120 bytes} [Mon Oct 10 11:48:25 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 10 11:48:25.503479 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb918aa4-ee7c-4623-9dfd-d54a526de982 None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:48:25.524182 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:48:25.604887 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2b7d79-e8ea-4a67-9e52-81578707dc99 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:48:25.619706 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb918aa4-ee7c-4623-9dfd-d54a526de982 None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:48:25.657148 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac853dd-2512-4a80-abe9-c545882bf5ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:48:25.657780 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac853dd-2512-4a80-abe9-c545882bf5ec None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:48:25.658056 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac853dd-2512-4a80-abe9-c545882bf5ec None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:48:25.658326 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac853dd-2512-4a80-abe9-c545882bf5ec None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:48:25.665033 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856b4ff6-8134-420e-b3f4-bfafc8580415 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:48:25.666279 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856b4ff6-8134-420e-b3f4-bfafc8580415 None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:48:25.666586 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856b4ff6-8134-420e-b3f4-bfafc8580415 None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:48:25.666899 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856b4ff6-8134-420e-b3f4-bfafc8580415 None admin] PATH_INFO: `/v3/roles` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:48:25.667614 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856b4ff6-8134-420e-b3f4-bfafc8580415 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:48:25.667916 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856b4ff6-8134-420e-b3f4-bfafc8580415 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:48:25.669358 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856b4ff6-8134-420e-b3f4-bfafc8580415 None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:48:25.679245 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 533/1093] 10.222.0.46 () {56 vars in 1115 bytes} [Mon Oct 10 11:48:25 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 11:48:25.688839 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856b4ff6-8134-420e-b3f4-bfafc8580415 None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:48:25.712956 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:48:25.760299 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac853dd-2512-4a80-abe9-c545882bf5ec None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:48:25.762864 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac853dd-2512-4a80-abe9-c545882bf5ec None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:48:25.822497 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:48:25.855732 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856b4ff6-8134-420e-b3f4-bfafc8580415 None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:48:25.889780 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 561/1094] 10.222.0.46 () {58 vars in 1282 bytes} [Mon Oct 10 11:48:25 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 423 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 10 11:48:25.895788 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac853dd-2512-4a80-abe9-c545882bf5ec None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:48:25.904641 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2410bb2e-6830-4e8c-bd88-d7f61215a858 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:48:25.905664 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2410bb2e-6830-4e8c-bd88-d7f61215a858 None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:48:25.906131 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2410bb2e-6830-4e8c-bd88-d7f61215a858 None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:48:25.906536 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2410bb2e-6830-4e8c-bd88-d7f61215a858 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:48:25.922791 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2410bb2e-6830-4e8c-bd88-d7f61215a858 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:48:25.925174 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2410bb2e-6830-4e8c-bd88-d7f61215a858 None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:48:25.942613 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application [None req-2410bb2e-6830-4e8c-bd88-d7f61215a858 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:48:25.942613 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:48:25.942613 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:48:25.942613 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:48:25.942613 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:48:25.942613 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:48:25.942613 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:48:25.942613 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:48:25.942613 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:48:25.942613 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:48:25.942613 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:48:25.942613 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:48:25.942613 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 10 11:48:25.942613 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 10 11:48:25.942613 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 10 11:48:25.942613 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 10 11:48:25.944608 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:48:25.944608 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:48:25.944608 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:48:25.944608 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:48:25.944608 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:48:25.944608 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:48:25.944608 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:48:25.944608 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application with Lock( Oct 10 11:48:25.944608 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:48:25.944608 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:48:25.944608 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:48:25.944608 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:48:25.944608 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:48:25.944608 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:48:25.944608 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:48:25.944608 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:48:25.944608 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 10 11:48:25.946127 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 10 11:48:25.946127 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:48:25.946127 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Oct 10 11:48:25.946127 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 534/1095] 10.222.0.46 () {56 vars in 1108 bytes} [Mon Oct 10 11:48:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 10 11:48:25.953323 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2410bb2e-6830-4e8c-bd88-d7f61215a858 None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:48:25.970415 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:48:26.012277 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac853dd-2512-4a80-abe9-c545882bf5ec None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:48:26.066369 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d83eb6c-7650-42aa-be65-a643ce711202 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:48:26.067316 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d83eb6c-7650-42aa-be65-a643ce711202 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:48:26.067610 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d83eb6c-7650-42aa-be65-a643ce711202 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:48:26.067888 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d83eb6c-7650-42aa-be65-a643ce711202 None admin] PATH_INFO: `/v3/projects/7d854c0679c94853b4c23aeb48cc8f9e` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:48:26.069504 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d83eb6c-7650-42aa-be65-a643ce711202 None admin] RBAC: Authorizing `identity:get_project(project_id=7d854c0679c94853b4c23aeb48cc8f9e)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:48:26.071019 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d83eb6c-7650-42aa-be65-a643ce711202 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:48:26.072904 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 562/1096] 10.222.0.46 () {56 vars in 1186 bytes} [Mon Oct 10 11:48:25 2022] GET /identity/v3/projects/7d854c0679c94853b4c23aeb48cc8f9e => generated 311 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 11:48:26.079552 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d83eb6c-7650-42aa-be65-a643ce711202 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:48:26.098631 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2410bb2e-6830-4e8c-bd88-d7f61215a858 None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:48:26.138503 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:48:26.144213 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be618f2-aa5e-4db8-b565-41199d9089bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:48:26.144850 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be618f2-aa5e-4db8-b565-41199d9089bb None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:48:26.145127 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be618f2-aa5e-4db8-b565-41199d9089bb None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:48:26.145399 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be618f2-aa5e-4db8-b565-41199d9089bb None admin] PATH_INFO: `/v3/domains` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:48:26.146070 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be618f2-aa5e-4db8-b565-41199d9089bb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:48:26.146369 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be618f2-aa5e-4db8-b565-41199d9089bb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:48:26.147825 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be618f2-aa5e-4db8-b565-41199d9089bb None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:48:26.168524 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 535/1097] 10.222.0.46 () {56 vars in 1109 bytes} [Mon Oct 10 11:48:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 11:48:26.170001 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be618f2-aa5e-4db8-b565-41199d9089bb None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:48:26.214315 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d83eb6c-7650-42aa-be65-a643ce711202 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:48:26.262978 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06ec818-f55d-4696-8bee-e9c60fbb817e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:48:26.263710 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06ec818-f55d-4696-8bee-e9c60fbb817e None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:48:26.264109 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06ec818-f55d-4696-8bee-e9c60fbb817e None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:48:26.264507 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06ec818-f55d-4696-8bee-e9c60fbb817e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:48:26.265364 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06ec818-f55d-4696-8bee-e9c60fbb817e None admin] RBAC: Adding query filter params (role.id=0d822a0674b6495fac623877a3b38025, scope.project.id=7d854c0679c94853b4c23aeb48cc8f9e, user.id=004b90e236b54d0fb4b6c0aba5fa7b53) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:48:26.265790 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06ec818-f55d-4696-8bee-e9c60fbb817e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:48:26.267460 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06ec818-f55d-4696-8bee-e9c60fbb817e None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:48:26.271042 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:48:26.275180 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 563/1098] 10.222.0.46 () {56 vars in 1374 bytes} [Mon Oct 10 11:48:26 2022] GET /identity/v3/role_assignments?user.id=004b90e236b54d0fb4b6c0aba5fa7b53&scope.project.id=7d854c0679c94853b4c23aeb48cc8f9e&role.id=0d822a0674b6495fac623877a3b38025 => generated 608 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 11:48:26.311704 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be618f2-aa5e-4db8-b565-41199d9089bb None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:48:26.356771 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec0a751-bdb8-4733-84e5-3f81cc52a54e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:48:26.357581 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec0a751-bdb8-4733-84e5-3f81cc52a54e None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:48:26.357959 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec0a751-bdb8-4733-84e5-3f81cc52a54e None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:48:26.358337 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec0a751-bdb8-4733-84e5-3f81cc52a54e None admin] PATH_INFO: `/v3/users/nova` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:48:26.367838 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec0a751-bdb8-4733-84e5-3f81cc52a54e None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:48:26.370156 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec0a751-bdb8-4733-84e5-3f81cc52a54e None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:48:26.384282 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application [None req-2ec0a751-bdb8-4733-84e5-3f81cc52a54e None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 10 11:48:26.384282 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:48:26.384282 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:48:26.384282 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:48:26.384282 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:48:26.384282 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:48:26.384282 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:48:26.384282 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:48:26.384282 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:48:26.384282 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:48:26.384282 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:48:26.384282 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:48:26.384282 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 10 11:48:26.384282 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 10 11:48:26.384282 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 10 11:48:26.384282 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 10 11:48:26.386447 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:48:26.386447 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:48:26.386447 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 10 11:48:26.386447 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 10 11:48:26.386447 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 10 11:48:26.386447 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 10 11:48:26.386447 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:48:26.386447 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:48:26.386447 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:48:26.386447 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:48:26.386447 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:48:26.386447 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application with Lock( Oct 10 11:48:26.386447 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:48:26.386447 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:48:26.386447 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:48:26.386447 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:48:26.386447 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:48:26.386447 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:48:26.388583 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:48:26.388583 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:48:26.388583 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 10 11:48:26.388583 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 10 11:48:26.388583 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 10 11:48:26.388583 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 10 11:48:26.388583 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 10 11:48:26.388583 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 10 11:48:26.388583 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 10 11:48:26.388583 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Oct 10 11:48:26.388583 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 536/1099] 10.222.0.46 () {56 vars in 1093 bytes} [Mon Oct 10 11:48:26 2022] GET /identity/v3/users/nova => generated 82 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 10 11:48:26.396573 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06ec818-f55d-4696-8bee-e9c60fbb817e None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:48:26.536132 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06ec818-f55d-4696-8bee-e9c60fbb817e None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:48:26.576688 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8db12c0-49dc-4e6b-9f1f-ebb37ee5a12f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:48:26.577480 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8db12c0-49dc-4e6b-9f1f-ebb37ee5a12f None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:48:26.577851 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8db12c0-49dc-4e6b-9f1f-ebb37ee5a12f None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:48:26.578219 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8db12c0-49dc-4e6b-9f1f-ebb37ee5a12f None admin] PATH_INFO: `/v3/users` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:48:26.579179 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8db12c0-49dc-4e6b-9f1f-ebb37ee5a12f None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:48:26.579586 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8db12c0-49dc-4e6b-9f1f-ebb37ee5a12f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:48:26.581256 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8db12c0-49dc-4e6b-9f1f-ebb37ee5a12f None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:48:26.603638 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 564/1100] 10.222.0.46 () {56 vars in 1133 bytes} [Mon Oct 10 11:48:26 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:48:26.613107 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec0a751-bdb8-4733-84e5-3f81cc52a54e None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:48:26.707219 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:48:26.767499 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec0a751-bdb8-4733-84e5-3f81cc52a54e None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:48:26.805870 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8d876b-c74c-4469-a021-9a6f8c7525a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:48:26.806957 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8d876b-c74c-4469-a021-9a6f8c7525a5 None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:48:26.807237 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8d876b-c74c-4469-a021-9a6f8c7525a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:48:26.807507 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8d876b-c74c-4469-a021-9a6f8c7525a5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:48:26.884671 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8d876b-c74c-4469-a021-9a6f8c7525a5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:48:26.886132 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8d876b-c74c-4469-a021-9a6f8c7525a5 None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:48:26.943959 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:48:26.998601 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 537/1101] 10.222.0.46 () {58 vars in 1282 bytes} [Mon Oct 10 11:48:26 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 10 11:48:27.005288 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8db12c0-49dc-4e6b-9f1f-ebb37ee5a12f None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:48:27.110030 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:48:27.166038 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8db12c0-49dc-4e6b-9f1f-ebb37ee5a12f None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:48:27.222358 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91307d95-fb2d-4f70-b41c-9657aef4bc4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:48:27.223448 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91307d95-fb2d-4f70-b41c-9657aef4bc4b None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:48:27.223948 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91307d95-fb2d-4f70-b41c-9657aef4bc4b None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:48:27.224446 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91307d95-fb2d-4f70-b41c-9657aef4bc4b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:48:27.246054 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91307d95-fb2d-4f70-b41c-9657aef4bc4b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:48:27.248820 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91307d95-fb2d-4f70-b41c-9657aef4bc4b None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:48:27.275211 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application [None req-91307d95-fb2d-4f70-b41c-9657aef4bc4b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:48:27.275211 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:48:27.275211 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:48:27.275211 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:48:27.275211 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:48:27.275211 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:48:27.275211 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:48:27.275211 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:48:27.275211 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:48:27.275211 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:48:27.275211 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:48:27.275211 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:48:27.275211 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 10 11:48:27.275211 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 10 11:48:27.275211 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 10 11:48:27.275211 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 10 11:48:27.277071 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:48:27.277071 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:48:27.277071 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:48:27.277071 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:48:27.277071 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:48:27.277071 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:48:27.277071 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:48:27.277071 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application with Lock( Oct 10 11:48:27.277071 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:48:27.277071 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:48:27.277071 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:48:27.277071 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:48:27.277071 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:48:27.277071 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:48:27.277071 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:48:27.277071 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:48:27.277071 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 10 11:48:27.279270 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 10 11:48:27.279270 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:48:27.279270 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Oct 10 11:48:27.279270 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 565/1102] 10.222.0.46 () {56 vars in 1108 bytes} [Mon Oct 10 11:48:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:48:27.283740 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8d876b-c74c-4469-a021-9a6f8c7525a5 None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:48:27.414444 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8d876b-c74c-4469-a021-9a6f8c7525a5 None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:48:27.457704 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b466a7a5-e536-4222-a119-795ce83df00f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:48:27.458339 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b466a7a5-e536-4222-a119-795ce83df00f None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:48:27.458618 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b466a7a5-e536-4222-a119-795ce83df00f None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:48:27.458921 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b466a7a5-e536-4222-a119-795ce83df00f None admin] PATH_INFO: `/v3/domains` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:48:27.459597 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b466a7a5-e536-4222-a119-795ce83df00f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:48:27.459903 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b466a7a5-e536-4222-a119-795ce83df00f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:48:27.465269 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b466a7a5-e536-4222-a119-795ce83df00f None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:48:27.489521 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 538/1103] 10.222.0.46 () {56 vars in 1109 bytes} [Mon Oct 10 11:48:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:48:27.490711 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91307d95-fb2d-4f70-b41c-9657aef4bc4b None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:48:27.669447 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91307d95-fb2d-4f70-b41c-9657aef4bc4b None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:48:27.689049 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:48:27.728333 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b13323-4daa-4156-8091-7503383bfab8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:48:27.729442 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b13323-4daa-4156-8091-7503383bfab8 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:48:27.730115 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b13323-4daa-4156-8091-7503383bfab8 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:48:27.730692 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b13323-4daa-4156-8091-7503383bfab8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:48:27.744221 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b13323-4daa-4156-8091-7503383bfab8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:48:27.746741 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b13323-4daa-4156-8091-7503383bfab8 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:48:27.769146 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application [None req-e8b13323-4daa-4156-8091-7503383bfab8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 10 11:48:27.769146 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:48:27.769146 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:48:27.769146 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:48:27.769146 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:48:27.769146 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:48:27.769146 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:48:27.769146 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:48:27.769146 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:48:27.769146 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:48:27.769146 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:48:27.769146 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:48:27.769146 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 10 11:48:27.769146 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 10 11:48:27.769146 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 10 11:48:27.770494 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 10 11:48:27.770494 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:48:27.770494 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:48:27.770494 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:48:27.770494 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:48:27.770494 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:48:27.770494 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:48:27.770494 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:48:27.770494 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application with Lock( Oct 10 11:48:27.770494 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:48:27.770494 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:48:27.770494 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:48:27.770494 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:48:27.770494 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:48:27.770494 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:48:27.770494 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:48:27.770494 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:48:27.771638 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 10 11:48:27.771638 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 10 11:48:27.771638 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 10 11:48:27.771638 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 10 11:48:27.771638 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 10 11:48:27.771638 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 10 11:48:27.771638 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 10 11:48:27.771638 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Oct 10 11:48:27.771638 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 566/1104] 10.222.0.46 () {56 vars in 1111 bytes} [Mon Oct 10 11:48:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 10 11:48:27.781479 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b466a7a5-e536-4222-a119-795ce83df00f None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:48:27.808288 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:48:27.957459 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b466a7a5-e536-4222-a119-795ce83df00f None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:48:28.004991 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44732a7e-a268-4dfa-ae78-c486e394fd55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:48:28.005609 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44732a7e-a268-4dfa-ae78-c486e394fd55 None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:48:28.005893 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44732a7e-a268-4dfa-ae78-c486e394fd55 None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:48:28.006165 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44732a7e-a268-4dfa-ae78-c486e394fd55 None admin] PATH_INFO: `/v3/projects` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:48:28.006897 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44732a7e-a268-4dfa-ae78-c486e394fd55 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:48:28.007202 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44732a7e-a268-4dfa-ae78-c486e394fd55 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:48:28.008637 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44732a7e-a268-4dfa-ae78-c486e394fd55 None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:48:28.022794 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 539/1105] 10.222.0.46 () {56 vars in 1148 bytes} [Mon Oct 10 11:48:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:48:28.030166 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b13323-4daa-4156-8091-7503383bfab8 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:48:28.162390 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b13323-4daa-4156-8091-7503383bfab8 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:48:28.202191 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ab541f-bcba-4719-9218-5cc7e9bdc2ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:48:28.203000 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ab541f-bcba-4719-9218-5cc7e9bdc2ac None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:48:28.203460 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ab541f-bcba-4719-9218-5cc7e9bdc2ac None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:48:28.203926 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ab541f-bcba-4719-9218-5cc7e9bdc2ac None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:48:28.204966 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ab541f-bcba-4719-9218-5cc7e9bdc2ac None admin] RBAC: Adding query filter params (role.id=66c72919d8744554bc2e7c3dcc683e1f, scope.project.id=4ad3772f7b6a47c28b7ba37a9316a3c1, user.id=5f130bbb69e546ce99b4e14de81d7e54) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:48:28.205407 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ab541f-bcba-4719-9218-5cc7e9bdc2ac None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:48:28.206879 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ab541f-bcba-4719-9218-5cc7e9bdc2ac None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:48:28.214442 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 567/1106] 10.222.0.46 () {56 vars in 1374 bytes} [Mon Oct 10 11:48:28 2022] GET /identity/v3/role_assignments?user.id=5f130bbb69e546ce99b4e14de81d7e54&scope.project.id=4ad3772f7b6a47c28b7ba37a9316a3c1&role.id=66c72919d8744554bc2e7c3dcc683e1f => generated 608 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 11:48:28.560351 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4ed7bf-73ae-4267-9d64-cba48e3f0a0e None None] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:48:28.560719 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4ed7bf-73ae-4267-9d64-cba48e3f0a0e None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:48:28.561078 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4ed7bf-73ae-4267-9d64-cba48e3f0a0e None None] PATH_INFO: `/` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:48:28.563478 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 540/1107] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 10 11:48:28 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:48:28.569934 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779427ef-db2d-4b8d-be4b-abb22405f8bf None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:48:28.570224 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779427ef-db2d-4b8d-be4b-abb22405f8bf None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:48:28.570493 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779427ef-db2d-4b8d-be4b-abb22405f8bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:48:28.593185 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-779427ef-db2d-4b8d-be4b-abb22405f8bf None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:48:28.594496 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-779427ef-db2d-4b8d-be4b-abb22405f8bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:48:28.625816 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:48:28.683417 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 568/1108] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 10 11:48:28 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 10 11:48:28.958299 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4ed7bf-73ae-4267-9d64-cba48e3f0a0e None None] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:48:29.064984 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:48:29.100434 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4ed7bf-73ae-4267-9d64-cba48e3f0a0e None None] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:48:29.141085 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1e3290-148d-4f4c-826f-2d1b2a64c114 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:48:29.141835 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1e3290-148d-4f4c-826f-2d1b2a64c114 None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:48:29.142227 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1e3290-148d-4f4c-826f-2d1b2a64c114 None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:48:29.142762 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1e3290-148d-4f4c-826f-2d1b2a64c114 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:48:29.144218 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1e3290-148d-4f4c-826f-2d1b2a64c114 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:48:29.145943 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1e3290-148d-4f4c-826f-2d1b2a64c114 None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:48:29.148276 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 541/1109] 10.222.0.46 () {56 vars in 1108 bytes} [Mon Oct 10 11:48:28 2022] GET /identity/v3/domains/default => generated 199 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:48:29.156802 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779427ef-db2d-4b8d-be4b-abb22405f8bf None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:48:29.307868 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779427ef-db2d-4b8d-be4b-abb22405f8bf None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:48:29.369273 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b6f135-1900-4255-90cc-bfd0e137839c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:48:29.370317 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b6f135-1900-4255-90cc-bfd0e137839c None admin] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:48:29.370843 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b6f135-1900-4255-90cc-bfd0e137839c None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:48:29.371333 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b6f135-1900-4255-90cc-bfd0e137839c None admin] PATH_INFO: `/v3/users` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:48:29.372388 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b6f135-1900-4255-90cc-bfd0e137839c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:48:29.375109 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b6f135-1900-4255-90cc-bfd0e137839c None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:48:29.505564 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 569/1110] 10.222.0.46 () {60 vars in 1132 bytes} [Mon Oct 10 11:48:29 2022] POST /identity/v3/users => generated 286 bytes in 350 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 11:48:31.779711 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2b8660-29dc-44e6-986b-f1f2e8ecc190 None None] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:48:31.780248 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2b8660-29dc-44e6-986b-f1f2e8ecc190 None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:48:31.780248 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2b8660-29dc-44e6-986b-f1f2e8ecc190 None None] PATH_INFO: `/` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:48:31.781536 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 542/1111] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 10 11:48:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:48:31.787339 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea655ca-6c7b-49fc-8df9-b02e158311b9 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:48:31.788056 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea655ca-6c7b-49fc-8df9-b02e158311b9 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:48:31.788056 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea655ca-6c7b-49fc-8df9-b02e158311b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:48:31.806433 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-3ea655ca-6c7b-49fc-8df9-b02e158311b9 None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:48:31.807690 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-3ea655ca-6c7b-49fc-8df9-b02e158311b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:48:31.823493 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:48:31.879778 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 570/1112] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 10 11:48:31 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 10 11:48:32.205244 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2b8660-29dc-44e6-986b-f1f2e8ecc190 None None] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:48:32.316735 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2b8660-29dc-44e6-986b-f1f2e8ecc190 None None] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:48:32.352266 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf08934d-9254-4144-97c1-610bc127737a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:48:32.353414 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf08934d-9254-4144-97c1-610bc127737a None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:48:32.353697 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf08934d-9254-4144-97c1-610bc127737a None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:48:32.353971 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf08934d-9254-4144-97c1-610bc127737a None admin] PATH_INFO: `/v3/roles/6762beebc2bf4e6f804cbbe55a475e70` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:48:32.355417 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf08934d-9254-4144-97c1-610bc127737a None admin] RBAC: Authorizing `identity:get_role(role_id=6762beebc2bf4e6f804cbbe55a475e70)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:48:32.356795 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf08934d-9254-4144-97c1-610bc127737a None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:48:32.358144 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 543/1113] 10.222.0.46 () {56 vars in 1177 bytes} [Mon Oct 10 11:48:32 2022] GET /identity/v3/roles/6762beebc2bf4e6f804cbbe55a475e70 => generated 223 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:48:32.364244 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea655ca-6c7b-49fc-8df9-b02e158311b9 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:48:32.472780 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea655ca-6c7b-49fc-8df9-b02e158311b9 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:48:32.532164 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec800a98-faf3-4db5-8621-ecd6137876ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:48:32.533324 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec800a98-faf3-4db5-8621-ecd6137876ed None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:48:32.533933 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec800a98-faf3-4db5-8621-ecd6137876ed None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:48:32.534439 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec800a98-faf3-4db5-8621-ecd6137876ed None admin] PATH_INFO: `/v3/users/eec7785efaff4838a39754ef6e875e7e` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:48:32.554020 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec800a98-faf3-4db5-8621-ecd6137876ed None admin] RBAC: Authorizing `identity:get_user(user_id=eec7785efaff4838a39754ef6e875e7e)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:48:32.556416 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec800a98-faf3-4db5-8621-ecd6137876ed None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:48:32.558405 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 571/1114] 10.222.0.46 () {56 vars in 1177 bytes} [Mon Oct 10 11:48:32 2022] GET /identity/v3/users/eec7785efaff4838a39754ef6e875e7e => generated 286 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:48:32.565029 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf08934d-9254-4144-97c1-610bc127737a None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:48:32.578651 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:48:32.679569 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf08934d-9254-4144-97c1-610bc127737a None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:48:32.714188 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598ec31e-6076-4f70-895a-1411fa54d237 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:48:32.714838 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598ec31e-6076-4f70-895a-1411fa54d237 None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:48:32.715122 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598ec31e-6076-4f70-895a-1411fa54d237 None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:48:32.715399 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598ec31e-6076-4f70-895a-1411fa54d237 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:48:32.796897 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598ec31e-6076-4f70-895a-1411fa54d237 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:48:32.798362 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598ec31e-6076-4f70-895a-1411fa54d237 None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:48:32.810324 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:48:32.920644 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 544/1115] 10.222.0.46 () {58 vars in 1282 bytes} [Mon Oct 10 11:48:32 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 10 11:48:32.927223 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec800a98-faf3-4db5-8621-ecd6137876ed None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:48:32.939931 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:48:33.040281 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec800a98-faf3-4db5-8621-ecd6137876ed None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:48:33.076512 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6b694f-4e81-43ee-8704-88791f571afd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:48:33.077076 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6b694f-4e81-43ee-8704-88791f571afd None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:48:33.077359 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6b694f-4e81-43ee-8704-88791f571afd None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:48:33.077637 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6b694f-4e81-43ee-8704-88791f571afd None admin] PATH_INFO: `/v3/projects/7d854c0679c94853b4c23aeb48cc8f9e` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:48:33.078808 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b6b694f-4e81-43ee-8704-88791f571afd None admin] RBAC: Authorizing `identity:get_project(project_id=7d854c0679c94853b4c23aeb48cc8f9e)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:48:33.080244 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b6b694f-4e81-43ee-8704-88791f571afd None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:48:33.081934 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 572/1116] 10.222.0.46 () {56 vars in 1186 bytes} [Mon Oct 10 11:48:32 2022] GET /identity/v3/projects/7d854c0679c94853b4c23aeb48cc8f9e => generated 311 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 11:48:33.092952 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598ec31e-6076-4f70-895a-1411fa54d237 None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:48:33.202052 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598ec31e-6076-4f70-895a-1411fa54d237 None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:48:33.212691 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:48:33.238447 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bcb29b-2030-4103-a7a3-11a5f7791462 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:48:33.239110 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bcb29b-2030-4103-a7a3-11a5f7791462 None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:48:33.239395 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bcb29b-2030-4103-a7a3-11a5f7791462 None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:48:33.239672 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bcb29b-2030-4103-a7a3-11a5f7791462 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:48:33.240424 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44bcb29b-2030-4103-a7a3-11a5f7791462 None admin] RBAC: Adding query filter params (role.id=6762beebc2bf4e6f804cbbe55a475e70, scope.project.id=7d854c0679c94853b4c23aeb48cc8f9e, user.id=eec7785efaff4838a39754ef6e875e7e) {{(pid=81676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:48:33.240737 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44bcb29b-2030-4103-a7a3-11a5f7791462 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:48:33.242190 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44bcb29b-2030-4103-a7a3-11a5f7791462 None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:48:33.248683 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 545/1117] 10.222.0.46 () {56 vars in 1374 bytes} [Mon Oct 10 11:48:33 2022] GET /identity/v3/role_assignments?user.id=eec7785efaff4838a39754ef6e875e7e&scope.project.id=7d854c0679c94853b4c23aeb48cc8f9e&role.id=6762beebc2bf4e6f804cbbe55a475e70 => generated 259 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:48:36.206582 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb3dc6b-e6f9-4189-b7ae-46b3981c8e05 None None] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:48:36.207208 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb3dc6b-e6f9-4189-b7ae-46b3981c8e05 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:48:36.207208 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb3dc6b-e6f9-4189-b7ae-46b3981c8e05 None None] PATH_INFO: `/` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:48:36.208507 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 573/1118] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 10 11:48:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:48:36.215892 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721e973a-eda3-4fee-b3d6-f57cb975aa5e None None] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:48:36.216400 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721e973a-eda3-4fee-b3d6-f57cb975aa5e None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:48:36.216891 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721e973a-eda3-4fee-b3d6-f57cb975aa5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:48:36.246463 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.auth.core [None req-721e973a-eda3-4fee-b3d6-f57cb975aa5e None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:48:36.248611 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-721e973a-eda3-4fee-b3d6-f57cb975aa5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:48:36.360946 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 546/1119] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 10 11:48:36 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 10 11:48:36.633499 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb3dc6b-e6f9-4189-b7ae-46b3981c8e05 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:48:36.737163 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:48:36.765779 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb3dc6b-e6f9-4189-b7ae-46b3981c8e05 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:48:36.807295 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5931c3-d737-4944-b959-8f8abf74111b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:48:36.807954 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5931c3-d737-4944-b959-8f8abf74111b None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:48:36.808232 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5931c3-d737-4944-b959-8f8abf74111b None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:48:36.808499 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5931c3-d737-4944-b959-8f8abf74111b None admin] PATH_INFO: `/v3/roles/6762beebc2bf4e6f804cbbe55a475e70` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:48:36.809995 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5931c3-d737-4944-b959-8f8abf74111b None admin] RBAC: Authorizing `identity:get_role(role_id=6762beebc2bf4e6f804cbbe55a475e70)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:48:36.811451 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5931c3-d737-4944-b959-8f8abf74111b None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:48:36.812889 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 574/1120] 10.222.0.46 () {56 vars in 1177 bytes} [Mon Oct 10 11:48:36 2022] GET /identity/v3/roles/6762beebc2bf4e6f804cbbe55a475e70 => generated 223 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:48:36.819471 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721e973a-eda3-4fee-b3d6-f57cb975aa5e None None] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:48:36.932840 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721e973a-eda3-4fee-b3d6-f57cb975aa5e None None] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:48:36.934500 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:48:36.969542 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106e3f43-e12c-4019-a01e-12dc2c903dfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:48:36.970233 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106e3f43-e12c-4019-a01e-12dc2c903dfa None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:48:36.970561 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106e3f43-e12c-4019-a01e-12dc2c903dfa None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:48:36.970917 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106e3f43-e12c-4019-a01e-12dc2c903dfa None admin] PATH_INFO: `/v3/users/eec7785efaff4838a39754ef6e875e7e` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:48:36.972152 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106e3f43-e12c-4019-a01e-12dc2c903dfa None admin] RBAC: Authorizing `identity:get_user(user_id=eec7785efaff4838a39754ef6e875e7e)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:48:36.973637 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106e3f43-e12c-4019-a01e-12dc2c903dfa None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:48:36.975304 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 547/1121] 10.222.0.46 () {56 vars in 1177 bytes} [Mon Oct 10 11:48:36 2022] GET /identity/v3/users/eec7785efaff4838a39754ef6e875e7e => generated 286 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 11:48:36.984377 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5931c3-d737-4944-b959-8f8abf74111b None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:48:37.092116 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5931c3-d737-4944-b959-8f8abf74111b None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:48:37.127879 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1655dbe-7d4d-4431-a055-e1deb569b907 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:48:37.128513 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1655dbe-7d4d-4431-a055-e1deb569b907 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:48:37.128791 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1655dbe-7d4d-4431-a055-e1deb569b907 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:48:37.129063 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1655dbe-7d4d-4431-a055-e1deb569b907 None admin] PATH_INFO: `/v3/projects/7d854c0679c94853b4c23aeb48cc8f9e` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:48:37.130217 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1655dbe-7d4d-4431-a055-e1deb569b907 None admin] RBAC: Authorizing `identity:get_project(project_id=7d854c0679c94853b4c23aeb48cc8f9e)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:48:37.131768 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1655dbe-7d4d-4431-a055-e1deb569b907 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:48:37.133499 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 575/1122] 10.222.0.46 () {56 vars in 1186 bytes} [Mon Oct 10 11:48:36 2022] GET /identity/v3/projects/7d854c0679c94853b4c23aeb48cc8f9e => generated 311 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:48:37.143768 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106e3f43-e12c-4019-a01e-12dc2c903dfa None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:48:37.262144 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106e3f43-e12c-4019-a01e-12dc2c903dfa None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:48:37.311541 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1f28be-6b34-4fce-8226-43d9ae4c12ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:48:37.312684 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1f28be-6b34-4fce-8226-43d9ae4c12ac None admin] REQUEST_METHOD: `PUT` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:48:37.313177 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1f28be-6b34-4fce-8226-43d9ae4c12ac None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:48:37.313675 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1f28be-6b34-4fce-8226-43d9ae4c12ac None admin] PATH_INFO: `/v3/projects/7d854c0679c94853b4c23aeb48cc8f9e/users/eec7785efaff4838a39754ef6e875e7e/roles/6762beebc2bf4e6f804cbbe55a475e70` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:48:37.317190 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1f28be-6b34-4fce-8226-43d9ae4c12ac None admin] RBAC: Authorizing `identity:create_grant(project_id=7d854c0679c94853b4c23aeb48cc8f9e, user_id=eec7785efaff4838a39754ef6e875e7e, role_id=6762beebc2bf4e6f804cbbe55a475e70)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:48:37.319894 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1f28be-6b34-4fce-8226-43d9ae4c12ac None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:48:37.407926 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 548/1123] 10.222.0.46 () {58 vars in 1439 bytes} [Mon Oct 10 11:48:37 2022] PUT /identity/v3/projects/7d854c0679c94853b4c23aeb48cc8f9e/users/eec7785efaff4838a39754ef6e875e7e/roles/6762beebc2bf4e6f804cbbe55a475e70 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 11:48:39.603434 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2e8c4e-14fb-4dcf-89ce-fa0b5bdefe29 None None] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:48:39.604033 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2e8c4e-14fb-4dcf-89ce-fa0b5bdefe29 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:48:39.604033 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2e8c4e-14fb-4dcf-89ce-fa0b5bdefe29 None None] PATH_INFO: `/` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:48:39.605516 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 576/1124] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 10 11:48:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:48:39.611982 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8245766f-3d4a-4228-9c72-a5b78eb126fa None None] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:48:39.612799 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8245766f-3d4a-4228-9c72-a5b78eb126fa None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:48:39.612953 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8245766f-3d4a-4228-9c72-a5b78eb126fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:48:39.634660 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.auth.core [None req-8245766f-3d4a-4228-9c72-a5b78eb126fa None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:48:39.635931 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-8245766f-3d4a-4228-9c72-a5b78eb126fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:48:39.663614 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:48:39.714784 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 549/1125] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 10 11:48:39 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 10 11:48:39.983570 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2e8c4e-14fb-4dcf-89ce-fa0b5bdefe29 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:48:39.999907 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:48:40.095358 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2e8c4e-14fb-4dcf-89ce-fa0b5bdefe29 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:48:40.130780 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627e7c15-3f42-49aa-807b-1bbc99ec8986 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:48:40.131421 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627e7c15-3f42-49aa-807b-1bbc99ec8986 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:48:40.131695 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627e7c15-3f42-49aa-807b-1bbc99ec8986 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:48:40.131963 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627e7c15-3f42-49aa-807b-1bbc99ec8986 None admin] PATH_INFO: `/v3/roles/6762beebc2bf4e6f804cbbe55a475e70` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:48:40.133407 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627e7c15-3f42-49aa-807b-1bbc99ec8986 None admin] RBAC: Authorizing `identity:get_role(role_id=6762beebc2bf4e6f804cbbe55a475e70)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:48:40.134909 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627e7c15-3f42-49aa-807b-1bbc99ec8986 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:48:40.136267 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 577/1126] 10.222.0.46 () {56 vars in 1177 bytes} [Mon Oct 10 11:48:39 2022] GET /identity/v3/roles/6762beebc2bf4e6f804cbbe55a475e70 => generated 223 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 11:48:40.142229 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8245766f-3d4a-4228-9c72-a5b78eb126fa None None] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:48:40.257163 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8245766f-3d4a-4228-9c72-a5b78eb126fa None None] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:48:40.300788 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95d6e55-91b4-41db-8694-fd55a395cbb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:48:40.301589 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95d6e55-91b4-41db-8694-fd55a395cbb3 None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:48:40.301973 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95d6e55-91b4-41db-8694-fd55a395cbb3 None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:48:40.302335 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95d6e55-91b4-41db-8694-fd55a395cbb3 None admin] PATH_INFO: `/v3/users/eec7785efaff4838a39754ef6e875e7e` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:48:40.303742 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95d6e55-91b4-41db-8694-fd55a395cbb3 None admin] RBAC: Authorizing `identity:get_user(user_id=eec7785efaff4838a39754ef6e875e7e)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:48:40.305878 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95d6e55-91b4-41db-8694-fd55a395cbb3 None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:48:40.307674 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 550/1127] 10.222.0.46 () {56 vars in 1177 bytes} [Mon Oct 10 11:48:40 2022] GET /identity/v3/users/eec7785efaff4838a39754ef6e875e7e => generated 286 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:48:40.313779 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627e7c15-3f42-49aa-807b-1bbc99ec8986 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:48:40.406343 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:48:40.426414 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627e7c15-3f42-49aa-807b-1bbc99ec8986 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:48:40.463920 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0fe698-680f-4abb-915a-6e23d9fa3cea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:48:40.464645 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0fe698-680f-4abb-915a-6e23d9fa3cea None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:48:40.465067 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0fe698-680f-4abb-915a-6e23d9fa3cea None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:48:40.465424 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0fe698-680f-4abb-915a-6e23d9fa3cea None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:48:40.544369 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0fe698-680f-4abb-915a-6e23d9fa3cea None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:48:40.545847 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0fe698-680f-4abb-915a-6e23d9fa3cea None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:48:40.637324 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:48:40.659239 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 578/1128] 10.222.0.46 () {58 vars in 1282 bytes} [Mon Oct 10 11:48:40 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 10 11:48:40.665527 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95d6e55-91b4-41db-8694-fd55a395cbb3 None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:48:40.695029 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:48:40.769930 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95d6e55-91b4-41db-8694-fd55a395cbb3 None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:48:40.804395 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f18e5d8-8b6a-4c1c-b734-ad2cb5e565a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:48:40.804988 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f18e5d8-8b6a-4c1c-b734-ad2cb5e565a9 None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:48:40.805303 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f18e5d8-8b6a-4c1c-b734-ad2cb5e565a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:48:40.805612 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f18e5d8-8b6a-4c1c-b734-ad2cb5e565a9 None admin] PATH_INFO: `/v3/projects/7d854c0679c94853b4c23aeb48cc8f9e` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:48:40.806826 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f18e5d8-8b6a-4c1c-b734-ad2cb5e565a9 None admin] RBAC: Authorizing `identity:get_project(project_id=7d854c0679c94853b4c23aeb48cc8f9e)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:48:40.808297 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f18e5d8-8b6a-4c1c-b734-ad2cb5e565a9 None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:48:40.810002 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 551/1129] 10.222.0.46 () {56 vars in 1186 bytes} [Mon Oct 10 11:48:40 2022] GET /identity/v3/projects/7d854c0679c94853b4c23aeb48cc8f9e => generated 311 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 11:48:40.816358 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0fe698-680f-4abb-915a-6e23d9fa3cea None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:48:40.918573 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0fe698-680f-4abb-915a-6e23d9fa3cea None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:48:40.955629 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c06f4c-b276-4bf0-a49b-4c22a67d89e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:48:40.956236 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c06f4c-b276-4bf0-a49b-4c22a67d89e8 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:48:40.956509 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c06f4c-b276-4bf0-a49b-4c22a67d89e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:48:40.956778 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c06f4c-b276-4bf0-a49b-4c22a67d89e8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:48:40.957494 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c06f4c-b276-4bf0-a49b-4c22a67d89e8 None admin] RBAC: Adding query filter params (role.id=6762beebc2bf4e6f804cbbe55a475e70, scope.project.id=7d854c0679c94853b4c23aeb48cc8f9e, user.id=eec7785efaff4838a39754ef6e875e7e) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:48:40.957794 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c06f4c-b276-4bf0-a49b-4c22a67d89e8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:48:40.959240 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c06f4c-b276-4bf0-a49b-4c22a67d89e8 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:48:40.965736 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 579/1130] 10.222.0.46 () {56 vars in 1374 bytes} [Mon Oct 10 11:48:40 2022] GET /identity/v3/role_assignments?user.id=eec7785efaff4838a39754ef6e875e7e&scope.project.id=7d854c0679c94853b4c23aeb48cc8f9e&role.id=6762beebc2bf4e6f804cbbe55a475e70 => generated 608 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:48:43.415719 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bdf7fd-0f4e-4d49-8b2c-b1fc3fc7e185 None None] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:48:43.416262 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bdf7fd-0f4e-4d49-8b2c-b1fc3fc7e185 None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:48:43.416262 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bdf7fd-0f4e-4d49-8b2c-b1fc3fc7e185 None None] PATH_INFO: `/` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:48:43.417790 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 552/1131] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 10 11:48:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:48:43.424633 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7db40a-e2eb-442b-9029-2ad7c7a6eb54 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:48:43.425156 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7db40a-e2eb-442b-9029-2ad7c7a6eb54 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:48:43.425156 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7db40a-e2eb-442b-9029-2ad7c7a6eb54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:48:43.444391 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-3e7db40a-e2eb-442b-9029-2ad7c7a6eb54 None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:48:43.445592 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-3e7db40a-e2eb-442b-9029-2ad7c7a6eb54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:48:43.491065 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:48:43.518144 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 580/1132] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 10 11:48:43 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 10 11:48:43.792280 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bdf7fd-0f4e-4d49-8b2c-b1fc3fc7e185 None None] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:48:43.907300 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:48:43.915297 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bdf7fd-0f4e-4d49-8b2c-b1fc3fc7e185 None None] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:48:43.951965 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c487ae2-dfe9-4bac-b98a-2480f0edfc33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:48:43.952620 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c487ae2-dfe9-4bac-b98a-2480f0edfc33 None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:48:43.952896 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c487ae2-dfe9-4bac-b98a-2480f0edfc33 None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:48:43.953179 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c487ae2-dfe9-4bac-b98a-2480f0edfc33 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:48:43.954364 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c487ae2-dfe9-4bac-b98a-2480f0edfc33 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:48:43.955853 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c487ae2-dfe9-4bac-b98a-2480f0edfc33 None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:48:43.957472 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 553/1133] 10.222.0.46 () {56 vars in 1108 bytes} [Mon Oct 10 11:48:43 2022] GET /identity/v3/domains/default => generated 199 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:48:43.965797 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7db40a-e2eb-442b-9029-2ad7c7a6eb54 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:48:44.072654 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7db40a-e2eb-442b-9029-2ad7c7a6eb54 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:48:44.109636 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fd84fa-292d-4883-b721-8cb84329b51f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:48:44.110255 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fd84fa-292d-4883-b721-8cb84329b51f None admin] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:48:44.110541 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fd84fa-292d-4883-b721-8cb84329b51f None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:48:44.110852 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fd84fa-292d-4883-b721-8cb84329b51f None admin] PATH_INFO: `/v3/projects` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:48:44.111528 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7fd84fa-292d-4883-b721-8cb84329b51f None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:48:44.112968 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7fd84fa-292d-4883-b721-8cb84329b51f None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:48:44.249223 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 581/1134] 10.222.0.46 () {60 vars in 1141 bytes} [Mon Oct 10 11:48:43 2022] POST /identity/v3/projects => generated 311 bytes in 284 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 11:48:46.536266 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caac2720-1d62-402b-9002-ddb65e27f582 None None] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:48:46.536833 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caac2720-1d62-402b-9002-ddb65e27f582 None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:48:46.536833 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caac2720-1d62-402b-9002-ddb65e27f582 None None] PATH_INFO: `/` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:48:46.538169 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 554/1135] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 10 11:48:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 10 11:48:46.547150 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e39d7f-4ce9-4bbd-9182-3c1d35fc09f1 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:48:46.547684 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e39d7f-4ce9-4bbd-9182-3c1d35fc09f1 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:48:46.547684 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e39d7f-4ce9-4bbd-9182-3c1d35fc09f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:48:46.567440 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-f7e39d7f-4ce9-4bbd-9182-3c1d35fc09f1 None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:48:46.568693 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-f7e39d7f-4ce9-4bbd-9182-3c1d35fc09f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:48:46.592865 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:48:46.641313 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 582/1136] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 10 11:48:46 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 10 11:48:46.916768 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caac2720-1d62-402b-9002-ddb65e27f582 None None] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:48:47.025294 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caac2720-1d62-402b-9002-ddb65e27f582 None None] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:48:47.062320 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ea79a4-7243-4dd8-9956-e4e21d26b784 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:48:47.062990 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ea79a4-7243-4dd8-9956-e4e21d26b784 None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:48:47.063275 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ea79a4-7243-4dd8-9956-e4e21d26b784 None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:48:47.063557 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ea79a4-7243-4dd8-9956-e4e21d26b784 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:48:47.064737 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ea79a4-7243-4dd8-9956-e4e21d26b784 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:48:47.066224 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ea79a4-7243-4dd8-9956-e4e21d26b784 None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:48:47.067886 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 555/1137] 10.222.0.46 () {56 vars in 1108 bytes} [Mon Oct 10 11:48:46 2022] GET /identity/v3/domains/default => generated 199 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 11:48:47.074434 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e39d7f-4ce9-4bbd-9182-3c1d35fc09f1 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:48:47.184396 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e39d7f-4ce9-4bbd-9182-3c1d35fc09f1 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:48:47.225952 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056f4ade-9a11-471a-8275-4d9163cc1131 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:48:47.226589 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056f4ade-9a11-471a-8275-4d9163cc1131 None admin] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:48:47.226903 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056f4ade-9a11-471a-8275-4d9163cc1131 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:48:47.227181 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056f4ade-9a11-471a-8275-4d9163cc1131 None admin] PATH_INFO: `/v3/users` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:48:47.227878 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056f4ade-9a11-471a-8275-4d9163cc1131 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:48:47.229356 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056f4ade-9a11-471a-8275-4d9163cc1131 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:48:47.333264 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 583/1138] 10.222.0.46 () {60 vars in 1132 bytes} [Mon Oct 10 11:48:47 2022] POST /identity/v3/users => generated 286 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 11:48:49.634327 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16fb913-595a-4536-9f94-d203335795cb None None] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:48:49.634929 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16fb913-595a-4536-9f94-d203335795cb None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:48:49.634929 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16fb913-595a-4536-9f94-d203335795cb None None] PATH_INFO: `/` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:48:49.636234 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 556/1139] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 10 11:48:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:48:49.642071 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334c9a7b-90d0-4800-a636-47092d0e9480 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:48:49.642581 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334c9a7b-90d0-4800-a636-47092d0e9480 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:48:49.642699 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334c9a7b-90d0-4800-a636-47092d0e9480 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:48:49.664096 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-334c9a7b-90d0-4800-a636-47092d0e9480 None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:48:49.665296 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-334c9a7b-90d0-4800-a636-47092d0e9480 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:48:49.673635 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:48:49.733503 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 584/1140] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 10 11:48:49 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 10 11:48:49.999109 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16fb913-595a-4536-9f94-d203335795cb None None] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:48:50.030647 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:48:50.103490 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16fb913-595a-4536-9f94-d203335795cb None None] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:48:50.137198 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469364d8-5639-48c3-8a50-44e0f8ebe994 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:48:50.137832 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469364d8-5639-48c3-8a50-44e0f8ebe994 None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:48:50.138117 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469364d8-5639-48c3-8a50-44e0f8ebe994 None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:48:50.138394 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469364d8-5639-48c3-8a50-44e0f8ebe994 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:48:50.149234 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469364d8-5639-48c3-8a50-44e0f8ebe994 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 10 11:48:50.154305 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application [None req-469364d8-5639-48c3-8a50-44e0f8ebe994 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 10 11:48:50.154305 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:48:50.154305 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:48:50.154305 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:48:50.154305 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:48:50.154305 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:48:50.154305 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:48:50.154305 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:48:50.154305 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:48:50.154305 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:48:50.154305 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:48:50.154305 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:48:50.154305 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 10 11:48:50.154305 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 10 11:48:50.154305 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 10 11:48:50.154305 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 10 11:48:50.160539 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 10 11:48:50.160539 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 10 11:48:50.160539 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 10 11:48:50.160539 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 10 11:48:50.160539 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:48:50.160539 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:48:50.160539 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:48:50.160539 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:48:50.160539 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:48:50.160539 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:48:50.160539 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:48:50.160539 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application with Lock( Oct 10 11:48:50.160539 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:48:50.160539 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:48:50.160539 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:48:50.160539 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:48:50.162063 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:48:50.162063 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:48:50.162063 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:48:50.162063 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:48:50.162063 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 10 11:48:50.162063 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 10 11:48:50.162063 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 10 11:48:50.162063 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 10 11:48:50.162063 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 10 11:48:50.162063 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 10 11:48:50.162063 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 10 11:48:50.162063 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Oct 10 11:48:50.162063 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 557/1141] 10.222.0.46 () {56 vars in 1096 bytes} [Mon Oct 10 11:48:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:48:50.163634 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334c9a7b-90d0-4800-a636-47092d0e9480 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:48:50.268311 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334c9a7b-90d0-4800-a636-47092d0e9480 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:48:50.304135 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8c338e-a44b-49b0-8e2f-3b1e4f5d87a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:48:50.304783 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8c338e-a44b-49b0-8e2f-3b1e4f5d87a1 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:48:50.305075 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8c338e-a44b-49b0-8e2f-3b1e4f5d87a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:48:50.305354 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8c338e-a44b-49b0-8e2f-3b1e4f5d87a1 None admin] PATH_INFO: `/v3/roles` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:48:50.306025 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8c338e-a44b-49b0-8e2f-3b1e4f5d87a1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:48:50.306332 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8c338e-a44b-49b0-8e2f-3b1e4f5d87a1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:48:50.307732 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8c338e-a44b-49b0-8e2f-3b1e4f5d87a1 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:48:50.316918 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 585/1142] 10.222.0.46 () {56 vars in 1099 bytes} [Mon Oct 10 11:48:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:48:50.323630 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469364d8-5639-48c3-8a50-44e0f8ebe994 None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:48:50.422484 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:48:50.431747 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469364d8-5639-48c3-8a50-44e0f8ebe994 None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:48:50.466042 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109dcbd0-c5a2-4c44-be44-341ac9fcd355 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:48:50.466684 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109dcbd0-c5a2-4c44-be44-341ac9fcd355 None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:48:50.466974 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109dcbd0-c5a2-4c44-be44-341ac9fcd355 None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:48:50.467249 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109dcbd0-c5a2-4c44-be44-341ac9fcd355 None admin] PATH_INFO: `/v3/users/ab4bb237e93740dc8c60e6916b56a868` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:48:50.483543 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109dcbd0-c5a2-4c44-be44-341ac9fcd355 None admin] RBAC: Authorizing `identity:get_user(user_id=ab4bb237e93740dc8c60e6916b56a868)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:48:50.485036 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109dcbd0-c5a2-4c44-be44-341ac9fcd355 None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:48:50.486933 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 558/1143] 10.222.0.46 () {56 vars in 1177 bytes} [Mon Oct 10 11:48:50 2022] GET /identity/v3/users/ab4bb237e93740dc8c60e6916b56a868 => generated 286 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 11:48:50.493436 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8c338e-a44b-49b0-8e2f-3b1e4f5d87a1 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:48:50.497227 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:48:50.589945 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8c338e-a44b-49b0-8e2f-3b1e4f5d87a1 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:48:50.623095 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be728ff-8150-4dd6-a5dd-7086e35aed3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:48:50.623703 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be728ff-8150-4dd6-a5dd-7086e35aed3d None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:48:50.623979 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be728ff-8150-4dd6-a5dd-7086e35aed3d None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:48:50.624248 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be728ff-8150-4dd6-a5dd-7086e35aed3d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:48:50.692587 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be728ff-8150-4dd6-a5dd-7086e35aed3d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:48:50.694036 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be728ff-8150-4dd6-a5dd-7086e35aed3d None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:48:50.696881 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:48:50.794603 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 586/1144] 10.222.0.46 () {58 vars in 1282 bytes} [Mon Oct 10 11:48:50 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 10 11:48:50.800775 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109dcbd0-c5a2-4c44-be44-341ac9fcd355 None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:48:50.900405 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109dcbd0-c5a2-4c44-be44-341ac9fcd355 None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:48:50.937648 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ae57ed-9f70-4f28-9a9c-0eaa186dc719 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:48:50.938274 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ae57ed-9f70-4f28-9a9c-0eaa186dc719 None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:48:50.938549 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ae57ed-9f70-4f28-9a9c-0eaa186dc719 None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:48:50.938855 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ae57ed-9f70-4f28-9a9c-0eaa186dc719 None admin] PATH_INFO: `/v3/projects/2b6f5e13a0d54a8f8e75f8f4dc84b537` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:48:50.939974 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ae57ed-9f70-4f28-9a9c-0eaa186dc719 None admin] RBAC: Authorizing `identity:get_project(project_id=2b6f5e13a0d54a8f8e75f8f4dc84b537)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:48:50.941401 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ae57ed-9f70-4f28-9a9c-0eaa186dc719 None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:48:50.943059 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 559/1145] 10.222.0.46 () {56 vars in 1186 bytes} [Mon Oct 10 11:48:50 2022] GET /identity/v3/projects/2b6f5e13a0d54a8f8e75f8f4dc84b537 => generated 311 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:48:50.949963 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be728ff-8150-4dd6-a5dd-7086e35aed3d None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:48:51.052775 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be728ff-8150-4dd6-a5dd-7086e35aed3d None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:48:51.055325 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:48:51.087563 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa64f855-73ba-4113-bb00-b11ad3b9fa0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:48:51.088180 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa64f855-73ba-4113-bb00-b11ad3b9fa0a None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:48:51.088459 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa64f855-73ba-4113-bb00-b11ad3b9fa0a None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:48:51.088732 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa64f855-73ba-4113-bb00-b11ad3b9fa0a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:48:51.089465 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa64f855-73ba-4113-bb00-b11ad3b9fa0a None admin] RBAC: Adding query filter params (role.id=0d822a0674b6495fac623877a3b38025, scope.project.id=2b6f5e13a0d54a8f8e75f8f4dc84b537, user.id=ab4bb237e93740dc8c60e6916b56a868) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:48:51.089773 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa64f855-73ba-4113-bb00-b11ad3b9fa0a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:48:51.091241 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa64f855-73ba-4113-bb00-b11ad3b9fa0a None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:48:51.097655 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 587/1146] 10.222.0.46 () {56 vars in 1374 bytes} [Mon Oct 10 11:48:50 2022] GET /identity/v3/role_assignments?user.id=ab4bb237e93740dc8c60e6916b56a868&scope.project.id=2b6f5e13a0d54a8f8e75f8f4dc84b537&role.id=0d822a0674b6495fac623877a3b38025 => generated 259 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:48:53.295015 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c144e19-4178-44e1-8b4b-31b106f4999b None None] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:48:53.295689 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c144e19-4178-44e1-8b4b-31b106f4999b None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:48:53.295689 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c144e19-4178-44e1-8b4b-31b106f4999b None None] PATH_INFO: `/` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:48:53.296946 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 560/1147] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 10 11:48:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:48:53.302353 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79c6dbf-2acc-404f-9575-54110525a8c1 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:48:53.302822 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79c6dbf-2acc-404f-9575-54110525a8c1 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:48:53.303035 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79c6dbf-2acc-404f-9575-54110525a8c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:48:53.321526 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-f79c6dbf-2acc-404f-9575-54110525a8c1 None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:48:53.322774 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-f79c6dbf-2acc-404f-9575-54110525a8c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:48:53.389759 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 588/1148] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 10 11:48:53 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 10 11:48:53.658103 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c144e19-4178-44e1-8b4b-31b106f4999b None None] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:48:53.678066 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:48:53.762758 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c144e19-4178-44e1-8b4b-31b106f4999b None None] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:48:53.797736 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e723a2-377c-4e38-bc02-699a562854f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:48:53.798368 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e723a2-377c-4e38-bc02-699a562854f1 None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:48:53.798649 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e723a2-377c-4e38-bc02-699a562854f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:48:53.798958 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e723a2-377c-4e38-bc02-699a562854f1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:48:53.809160 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e723a2-377c-4e38-bc02-699a562854f1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 10 11:48:53.813991 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application [None req-12e723a2-377c-4e38-bc02-699a562854f1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 10 11:48:53.813991 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:48:53.813991 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:48:53.813991 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:48:53.813991 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:48:53.813991 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:48:53.813991 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:48:53.813991 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:48:53.813991 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:48:53.813991 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:48:53.813991 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:48:53.813991 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:48:53.813991 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 10 11:48:53.813991 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 10 11:48:53.813991 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 10 11:48:53.813991 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 10 11:48:53.815891 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 10 11:48:53.815891 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 10 11:48:53.815891 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 10 11:48:53.815891 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 10 11:48:53.815891 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:48:53.815891 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:48:53.815891 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:48:53.815891 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:48:53.815891 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:48:53.815891 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:48:53.815891 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:48:53.815891 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application with Lock( Oct 10 11:48:53.815891 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:48:53.815891 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:48:53.815891 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:48:53.815891 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:48:53.817669 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:48:53.817669 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:48:53.817669 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:48:53.817669 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:48:53.817669 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 10 11:48:53.817669 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 10 11:48:53.817669 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 10 11:48:53.817669 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 10 11:48:53.817669 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 10 11:48:53.817669 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 10 11:48:53.817669 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 10 11:48:53.817669 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Oct 10 11:48:53.817669 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 561/1149] 10.222.0.46 () {56 vars in 1096 bytes} [Mon Oct 10 11:48:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:48:53.822855 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79c6dbf-2acc-404f-9575-54110525a8c1 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:48:53.917176 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:48:53.921086 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79c6dbf-2acc-404f-9575-54110525a8c1 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:48:53.955272 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666bc6a4-452f-450e-bcb5-fc9471286588 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:48:53.955891 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666bc6a4-452f-450e-bcb5-fc9471286588 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:48:53.956501 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666bc6a4-452f-450e-bcb5-fc9471286588 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:48:53.956782 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666bc6a4-452f-450e-bcb5-fc9471286588 None admin] PATH_INFO: `/v3/roles` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:48:53.957458 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666bc6a4-452f-450e-bcb5-fc9471286588 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:48:53.957766 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666bc6a4-452f-450e-bcb5-fc9471286588 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:48:53.959171 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666bc6a4-452f-450e-bcb5-fc9471286588 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:48:53.967386 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 589/1150] 10.222.0.46 () {56 vars in 1099 bytes} [Mon Oct 10 11:48:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:48:53.973386 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e723a2-377c-4e38-bc02-699a562854f1 None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:48:54.054625 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:48:54.076672 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e723a2-377c-4e38-bc02-699a562854f1 None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:48:54.112287 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc85429c-a203-4f7b-9cec-5190754a0e4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:48:54.112923 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc85429c-a203-4f7b-9cec-5190754a0e4a None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:48:54.113207 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc85429c-a203-4f7b-9cec-5190754a0e4a None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:48:54.113485 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc85429c-a203-4f7b-9cec-5190754a0e4a None admin] PATH_INFO: `/v3/users/ab4bb237e93740dc8c60e6916b56a868` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:48:54.114640 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc85429c-a203-4f7b-9cec-5190754a0e4a None admin] RBAC: Authorizing `identity:get_user(user_id=ab4bb237e93740dc8c60e6916b56a868)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:48:54.116113 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc85429c-a203-4f7b-9cec-5190754a0e4a None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:48:54.118602 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 562/1151] 10.222.0.46 () {56 vars in 1177 bytes} [Mon Oct 10 11:48:53 2022] GET /identity/v3/users/ab4bb237e93740dc8c60e6916b56a868 => generated 286 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:48:54.125518 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666bc6a4-452f-450e-bcb5-fc9471286588 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:48:54.228402 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666bc6a4-452f-450e-bcb5-fc9471286588 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:48:54.263952 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ceb59ca-84fd-4c2b-ace2-ea693e37e07a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:48:54.264588 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ceb59ca-84fd-4c2b-ace2-ea693e37e07a None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:48:54.264864 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ceb59ca-84fd-4c2b-ace2-ea693e37e07a None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:48:54.265136 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ceb59ca-84fd-4c2b-ace2-ea693e37e07a None admin] PATH_INFO: `/v3/projects/2b6f5e13a0d54a8f8e75f8f4dc84b537` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:48:54.266252 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ceb59ca-84fd-4c2b-ace2-ea693e37e07a None admin] RBAC: Authorizing `identity:get_project(project_id=2b6f5e13a0d54a8f8e75f8f4dc84b537)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:48:54.267709 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ceb59ca-84fd-4c2b-ace2-ea693e37e07a None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:48:54.269352 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 590/1152] 10.222.0.46 () {56 vars in 1186 bytes} [Mon Oct 10 11:48:54 2022] GET /identity/v3/projects/2b6f5e13a0d54a8f8e75f8f4dc84b537 => generated 311 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:48:54.276129 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc85429c-a203-4f7b-9cec-5190754a0e4a None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:48:54.377559 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc85429c-a203-4f7b-9cec-5190754a0e4a None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:48:54.419564 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c6c219-73f2-4d4d-b7c2-2e40d48cfdbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:48:54.420253 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c6c219-73f2-4d4d-b7c2-2e40d48cfdbd None admin] REQUEST_METHOD: `PUT` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:48:54.420534 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c6c219-73f2-4d4d-b7c2-2e40d48cfdbd None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:48:54.420810 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c6c219-73f2-4d4d-b7c2-2e40d48cfdbd None admin] PATH_INFO: `/v3/projects/2b6f5e13a0d54a8f8e75f8f4dc84b537/users/ab4bb237e93740dc8c60e6916b56a868/roles/0d822a0674b6495fac623877a3b38025` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:48:54.422826 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c6c219-73f2-4d4d-b7c2-2e40d48cfdbd None admin] RBAC: Authorizing `identity:create_grant(project_id=2b6f5e13a0d54a8f8e75f8f4dc84b537, user_id=ab4bb237e93740dc8c60e6916b56a868, role_id=0d822a0674b6495fac623877a3b38025)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:48:54.424284 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c6c219-73f2-4d4d-b7c2-2e40d48cfdbd None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:48:54.516840 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 563/1153] 10.222.0.46 () {58 vars in 1439 bytes} [Mon Oct 10 11:48:54 2022] PUT /identity/v3/projects/2b6f5e13a0d54a8f8e75f8f4dc84b537/users/ab4bb237e93740dc8c60e6916b56a868/roles/0d822a0674b6495fac623877a3b38025 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 11:48:56.704781 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0c8ffb-23f3-45cb-8413-26edff1089d7 None None] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:48:56.705358 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0c8ffb-23f3-45cb-8413-26edff1089d7 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:48:56.705358 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0c8ffb-23f3-45cb-8413-26edff1089d7 None None] PATH_INFO: `/` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:48:56.706713 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 591/1154] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 10 11:48:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:48:56.715524 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523e6df6-8f1f-4746-89c0-722d4c2ce84e None None] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:48:56.716023 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523e6df6-8f1f-4746-89c0-722d4c2ce84e None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:48:56.716101 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523e6df6-8f1f-4746-89c0-722d4c2ce84e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:48:56.730624 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:48:56.736779 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.auth.core [None req-523e6df6-8f1f-4746-89c0-722d4c2ce84e None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:48:56.737912 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-523e6df6-8f1f-4746-89c0-722d4c2ce84e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:48:56.806902 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 564/1155] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 10 11:48:56 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 10 11:48:57.075847 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0c8ffb-23f3-45cb-8413-26edff1089d7 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:48:57.107816 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:48:57.184947 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0c8ffb-23f3-45cb-8413-26edff1089d7 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:48:57.222737 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee93247-c996-4336-8e44-fc5f2fcbacbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:48:57.223361 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee93247-c996-4336-8e44-fc5f2fcbacbf None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:48:57.223670 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee93247-c996-4336-8e44-fc5f2fcbacbf None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:48:57.224058 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee93247-c996-4336-8e44-fc5f2fcbacbf None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:48:57.235952 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee93247-c996-4336-8e44-fc5f2fcbacbf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 10 11:48:57.240839 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application [None req-dee93247-c996-4336-8e44-fc5f2fcbacbf None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 10 11:48:57.240839 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:48:57.240839 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:48:57.240839 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:48:57.240839 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:48:57.240839 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:48:57.240839 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:48:57.240839 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:48:57.240839 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:48:57.240839 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:48:57.240839 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:48:57.240839 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:48:57.240839 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 10 11:48:57.240839 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 10 11:48:57.240839 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 10 11:48:57.240839 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 10 11:48:57.243299 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 10 11:48:57.243299 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 10 11:48:57.243299 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 10 11:48:57.243299 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 10 11:48:57.243299 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:48:57.243299 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:48:57.243299 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:48:57.243299 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:48:57.243299 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:48:57.243299 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:48:57.243299 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:48:57.243299 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application with Lock( Oct 10 11:48:57.243299 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:48:57.243299 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:48:57.243299 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:48:57.243299 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:48:57.244852 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:48:57.244852 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:48:57.244852 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:48:57.244852 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:48:57.244852 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 10 11:48:57.244852 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 10 11:48:57.244852 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 10 11:48:57.244852 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 10 11:48:57.244852 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 10 11:48:57.244852 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 10 11:48:57.244852 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 10 11:48:57.244852 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Oct 10 11:48:57.244852 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 592/1156] 10.222.0.46 () {56 vars in 1096 bytes} [Mon Oct 10 11:48:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:48:57.248913 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523e6df6-8f1f-4746-89c0-722d4c2ce84e None None] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:48:57.352920 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523e6df6-8f1f-4746-89c0-722d4c2ce84e None None] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:48:57.387559 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659be822-584b-4b8a-b722-c56f79940aea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:48:57.388193 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659be822-584b-4b8a-b722-c56f79940aea None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:48:57.388476 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659be822-584b-4b8a-b722-c56f79940aea None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:48:57.388756 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659be822-584b-4b8a-b722-c56f79940aea None admin] PATH_INFO: `/v3/roles` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:48:57.389463 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659be822-584b-4b8a-b722-c56f79940aea None admin] RBAC: Adding query filter params (name=admin) {{(pid=81676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:48:57.389767 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659be822-584b-4b8a-b722-c56f79940aea None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:48:57.391264 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659be822-584b-4b8a-b722-c56f79940aea None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:48:57.394291 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:48:57.402866 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 565/1157] 10.222.0.46 () {56 vars in 1099 bytes} [Mon Oct 10 11:48:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:48:57.410599 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee93247-c996-4336-8e44-fc5f2fcbacbf None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:48:57.517025 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:48:57.531224 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee93247-c996-4336-8e44-fc5f2fcbacbf None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:48:57.570451 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00eea715-8551-4034-81de-91257d8c0031 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:48:57.571122 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00eea715-8551-4034-81de-91257d8c0031 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:48:57.571405 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00eea715-8551-4034-81de-91257d8c0031 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:48:57.571675 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00eea715-8551-4034-81de-91257d8c0031 None admin] PATH_INFO: `/v3/users/ab4bb237e93740dc8c60e6916b56a868` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:48:57.572832 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00eea715-8551-4034-81de-91257d8c0031 None admin] RBAC: Authorizing `identity:get_user(user_id=ab4bb237e93740dc8c60e6916b56a868)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:48:57.574272 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00eea715-8551-4034-81de-91257d8c0031 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:48:57.575989 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 593/1158] 10.222.0.46 () {56 vars in 1177 bytes} [Mon Oct 10 11:48:57 2022] GET /identity/v3/users/ab4bb237e93740dc8c60e6916b56a868 => generated 286 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:48:57.583173 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659be822-584b-4b8a-b722-c56f79940aea None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:48:57.699424 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659be822-584b-4b8a-b722-c56f79940aea None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:48:57.738596 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d88b0b-0f97-4b92-bb47-8c0da5190236 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:48:57.739314 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d88b0b-0f97-4b92-bb47-8c0da5190236 None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:48:57.739525 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d88b0b-0f97-4b92-bb47-8c0da5190236 None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:48:57.739797 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d88b0b-0f97-4b92-bb47-8c0da5190236 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:48:57.811078 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:48:57.814856 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d88b0b-0f97-4b92-bb47-8c0da5190236 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:48:57.816369 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d88b0b-0f97-4b92-bb47-8c0da5190236 None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:48:57.914865 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 566/1159] 10.222.0.46 () {58 vars in 1282 bytes} [Mon Oct 10 11:48:57 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 10 11:48:57.921283 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00eea715-8551-4034-81de-91257d8c0031 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:48:58.026605 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00eea715-8551-4034-81de-91257d8c0031 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:48:58.062926 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61072ca-4651-4216-a49d-be1957e096f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:48:58.063565 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61072ca-4651-4216-a49d-be1957e096f8 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:48:58.063844 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61072ca-4651-4216-a49d-be1957e096f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:48:58.064126 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61072ca-4651-4216-a49d-be1957e096f8 None admin] PATH_INFO: `/v3/projects/2b6f5e13a0d54a8f8e75f8f4dc84b537` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:48:58.065254 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61072ca-4651-4216-a49d-be1957e096f8 None admin] RBAC: Authorizing `identity:get_project(project_id=2b6f5e13a0d54a8f8e75f8f4dc84b537)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:48:58.066713 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61072ca-4651-4216-a49d-be1957e096f8 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:48:58.068386 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 594/1160] 10.222.0.46 () {56 vars in 1186 bytes} [Mon Oct 10 11:48:57 2022] GET /identity/v3/projects/2b6f5e13a0d54a8f8e75f8f4dc84b537 => generated 311 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:48:58.075424 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d88b0b-0f97-4b92-bb47-8c0da5190236 None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:48:58.175898 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:48:58.180742 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d88b0b-0f97-4b92-bb47-8c0da5190236 None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:48:58.216609 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f4dce2-7fd6-4f44-9236-481acef2ae9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:48:58.217221 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f4dce2-7fd6-4f44-9236-481acef2ae9b None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:48:58.217511 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f4dce2-7fd6-4f44-9236-481acef2ae9b None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:48:58.217786 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f4dce2-7fd6-4f44-9236-481acef2ae9b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:48:58.218515 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f4dce2-7fd6-4f44-9236-481acef2ae9b None admin] RBAC: Adding query filter params (role.id=0d822a0674b6495fac623877a3b38025, scope.project.id=2b6f5e13a0d54a8f8e75f8f4dc84b537, user.id=ab4bb237e93740dc8c60e6916b56a868) {{(pid=81676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:48:58.218858 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f4dce2-7fd6-4f44-9236-481acef2ae9b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:48:58.220296 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f4dce2-7fd6-4f44-9236-481acef2ae9b None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:48:58.227698 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 567/1161] 10.222.0.46 () {56 vars in 1374 bytes} [Mon Oct 10 11:48:58 2022] GET /identity/v3/role_assignments?user.id=ab4bb237e93740dc8c60e6916b56a868&scope.project.id=2b6f5e13a0d54a8f8e75f8f4dc84b537&role.id=0d822a0674b6495fac623877a3b38025 => generated 608 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:49:00.441759 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab05bb6-ebaf-4b7a-86aa-4705c13e76dd None None] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:49:00.442608 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab05bb6-ebaf-4b7a-86aa-4705c13e76dd None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:49:00.442608 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab05bb6-ebaf-4b7a-86aa-4705c13e76dd None None] PATH_INFO: `/` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:49:00.443696 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 595/1162] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 10 11:49:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:49:00.451008 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e337c0-735a-4dd0-a771-fd9491df782c None None] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:49:00.451550 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e337c0-735a-4dd0-a771-fd9491df782c None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:49:00.451550 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e337c0-735a-4dd0-a771-fd9491df782c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:49:00.478985 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.auth.core [None req-d1e337c0-735a-4dd0-a771-fd9491df782c None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:49:00.481134 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-d1e337c0-735a-4dd0-a771-fd9491df782c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:49:00.558135 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 568/1163] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 10 11:49:00 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 10 11:49:00.831428 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab05bb6-ebaf-4b7a-86aa-4705c13e76dd None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:49:00.859095 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:49:00.938087 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab05bb6-ebaf-4b7a-86aa-4705c13e76dd None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:49:00.973178 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049fcdb4-13e6-4dcf-90e9-f5c2c78f0d35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:49:00.973785 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049fcdb4-13e6-4dcf-90e9-f5c2c78f0d35 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:49:00.974070 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049fcdb4-13e6-4dcf-90e9-f5c2c78f0d35 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:49:00.974347 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049fcdb4-13e6-4dcf-90e9-f5c2c78f0d35 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:49:01.042942 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049fcdb4-13e6-4dcf-90e9-f5c2c78f0d35 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:49:01.044405 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049fcdb4-13e6-4dcf-90e9-f5c2c78f0d35 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:49:01.068684 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:49:01.146145 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 596/1164] 10.222.0.46 () {58 vars in 1282 bytes} [Mon Oct 10 11:49:00 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 10 11:49:01.152339 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e337c0-735a-4dd0-a771-fd9491df782c None None] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:49:01.248490 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:49:01.257534 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e337c0-735a-4dd0-a771-fd9491df782c None None] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:49:01.294137 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41242be-532f-4c93-9547-c99d05a53606 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:49:01.294827 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41242be-532f-4c93-9547-c99d05a53606 None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:49:01.295115 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41242be-532f-4c93-9547-c99d05a53606 None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:49:01.295398 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41242be-532f-4c93-9547-c99d05a53606 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:49:01.303441 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41242be-532f-4c93-9547-c99d05a53606 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:49:01.305470 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41242be-532f-4c93-9547-c99d05a53606 None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:49:01.318770 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application [None req-a41242be-532f-4c93-9547-c99d05a53606 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Oct 10 11:49:01.318770 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:49:01.318770 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Oct 10 11:49:01.318770 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Oct 10 11:49:01.318770 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 10 11:49:01.318770 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 10 11:49:01.318770 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 10 11:49:01.318770 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 10 11:49:01.318770 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Oct 10 11:49:01.318770 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Oct 10 11:49:01.318770 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 10 11:49:01.318770 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Oct 10 11:49:01.318770 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:49:01.318770 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:49:01.318770 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:49:01.318770 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:49:01.318770 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:49:01.320809 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:49:01.320809 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:49:01.320809 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:49:01.320809 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:49:01.320809 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:49:01.320809 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:49:01.320809 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 10 11:49:01.320809 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 10 11:49:01.320809 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 10 11:49:01.320809 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 10 11:49:01.320809 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:49:01.320809 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:49:01.320809 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:49:01.320809 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:49:01.320809 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:49:01.320809 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:49:01.321980 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:49:01.321980 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application with Lock( Oct 10 11:49:01.321980 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:49:01.321980 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:49:01.321980 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:49:01.321980 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:49:01.321980 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:49:01.321980 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:49:01.321980 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:49:01.321980 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:49:01.321980 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Oct 10 11:49:01.321980 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 10 11:49:01.321980 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Oct 10 11:49:01.321980 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Oct 10 11:49:01.321980 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 569/1165] 10.222.0.46 () {56 vars in 1117 bytes} [Mon Oct 10 11:49:01 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:49:01.327178 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049fcdb4-13e6-4dcf-90e9-f5c2c78f0d35 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:49:01.463352 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049fcdb4-13e6-4dcf-90e9-f5c2c78f0d35 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:49:01.492463 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:49:01.506473 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ad5af4-de18-4c7c-82e0-5deaa3bc030a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:49:01.507150 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ad5af4-de18-4c7c-82e0-5deaa3bc030a None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:49:01.507474 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ad5af4-de18-4c7c-82e0-5deaa3bc030a None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:49:01.508909 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ad5af4-de18-4c7c-82e0-5deaa3bc030a None admin] PATH_INFO: `/v3/domains` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:49:01.508909 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ad5af4-de18-4c7c-82e0-5deaa3bc030a None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:49:01.508909 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ad5af4-de18-4c7c-82e0-5deaa3bc030a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:49:01.510099 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ad5af4-de18-4c7c-82e0-5deaa3bc030a None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:49:01.517965 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 597/1166] 10.222.0.46 () {56 vars in 1115 bytes} [Mon Oct 10 11:49:01 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 11:49:03.720726 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ff3674-11e6-45c4-a385-8933883cdf8f None None] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:49:03.721274 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ff3674-11e6-45c4-a385-8933883cdf8f None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:49:03.721274 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ff3674-11e6-45c4-a385-8933883cdf8f None None] PATH_INFO: `/` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:49:03.722622 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 570/1167] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 10 11:49:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:49:03.728478 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a824cc03-b694-490f-b92c-685b8bb5db83 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:49:03.728932 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a824cc03-b694-490f-b92c-685b8bb5db83 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:49:03.729088 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a824cc03-b694-490f-b92c-685b8bb5db83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:49:03.747783 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-a824cc03-b694-490f-b92c-685b8bb5db83 None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:49:03.748887 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-a824cc03-b694-490f-b92c-685b8bb5db83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:49:03.812786 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 598/1168] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 10 11:49:03 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 10 11:49:04.076999 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ff3674-11e6-45c4-a385-8933883cdf8f None None] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:49:04.183655 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ff3674-11e6-45c4-a385-8933883cdf8f None None] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:49:04.219644 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa071ff-b8f0-4f00-82b0-7e2c24809460 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:49:04.220268 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa071ff-b8f0-4f00-82b0-7e2c24809460 None admin] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:49:04.220550 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa071ff-b8f0-4f00-82b0-7e2c24809460 None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:49:04.220828 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa071ff-b8f0-4f00-82b0-7e2c24809460 None admin] PATH_INFO: `/v3/domains` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:49:04.221509 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa071ff-b8f0-4f00-82b0-7e2c24809460 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:49:04.223088 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa071ff-b8f0-4f00-82b0-7e2c24809460 None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:49:04.312317 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 571/1169] 10.222.0.46 () {60 vars in 1138 bytes} [Mon Oct 10 11:49:04 2022] POST /identity/v3/domains => generated 267 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 11:49:06.527079 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ed9a30-6b65-42ac-af76-2066d46746e5 None None] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:49:06.527606 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ed9a30-6b65-42ac-af76-2066d46746e5 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:49:06.527733 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ed9a30-6b65-42ac-af76-2066d46746e5 None None] PATH_INFO: `/` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:49:06.528986 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 599/1170] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 10 11:49:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:49:06.534552 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82a0bf5-cd27-41df-b652-5a4bd566053c None None] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:49:06.535023 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82a0bf5-cd27-41df-b652-5a4bd566053c None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:49:06.535116 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82a0bf5-cd27-41df-b652-5a4bd566053c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:49:06.553410 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.auth.core [None req-c82a0bf5-cd27-41df-b652-5a4bd566053c None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:49:06.555155 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-c82a0bf5-cd27-41df-b652-5a4bd566053c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:49:06.563219 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:49:06.626188 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 572/1171] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 10 11:49:06 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 10 11:49:06.894802 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ed9a30-6b65-42ac-af76-2066d46746e5 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:49:06.999800 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ed9a30-6b65-42ac-af76-2066d46746e5 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:49:07.018128 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:49:07.034810 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25699527-29eb-4f25-b861-7d4adbe03fe8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:49:07.035445 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25699527-29eb-4f25-b861-7d4adbe03fe8 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:49:07.035724 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25699527-29eb-4f25-b861-7d4adbe03fe8 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:49:07.036002 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25699527-29eb-4f25-b861-7d4adbe03fe8 None admin] PATH_INFO: `/v3/domains/bddd0eaba1f44112baa754610413b793` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:49:07.047406 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25699527-29eb-4f25-b861-7d4adbe03fe8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=bddd0eaba1f44112baa754610413b793)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:49:07.048848 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25699527-29eb-4f25-b861-7d4adbe03fe8 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:49:07.050788 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 600/1172] 10.222.0.46 () {56 vars in 1183 bytes} [Mon Oct 10 11:49:06 2022] GET /identity/v3/domains/bddd0eaba1f44112baa754610413b793 => generated 267 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:49:07.057195 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82a0bf5-cd27-41df-b652-5a4bd566053c None None] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:49:07.164563 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82a0bf5-cd27-41df-b652-5a4bd566053c None None] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:49:07.200099 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d30d5f-36c1-4e32-8d72-d417f2daee27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:49:07.200755 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d30d5f-36c1-4e32-8d72-d417f2daee27 None admin] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:49:07.201036 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d30d5f-36c1-4e32-8d72-d417f2daee27 None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:49:07.201310 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d30d5f-36c1-4e32-8d72-d417f2daee27 None admin] PATH_INFO: `/v3/projects` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:49:07.202001 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d30d5f-36c1-4e32-8d72-d417f2daee27 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:49:07.203517 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d30d5f-36c1-4e32-8d72-d417f2daee27 None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:49:07.219583 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:49:07.287738 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 573/1173] 10.222.0.46 () {60 vars in 1141 bytes} [Mon Oct 10 11:49:07 2022] POST /identity/v3/projects => generated 361 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 11:49:09.521169 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4879f0c-05cd-4d13-bbd0-795dcb3ba5dd None None] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:49:09.521752 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4879f0c-05cd-4d13-bbd0-795dcb3ba5dd None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:49:09.521752 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4879f0c-05cd-4d13-bbd0-795dcb3ba5dd None None] PATH_INFO: `/` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:49:09.523215 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 601/1174] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 10 11:49:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:49:09.528525 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7870b0d6-6a65-4a36-ba6e-57f08c96ce3f None None] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:49:09.529145 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7870b0d6-6a65-4a36-ba6e-57f08c96ce3f None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:49:09.529145 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7870b0d6-6a65-4a36-ba6e-57f08c96ce3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:49:09.547850 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.auth.core [None req-7870b0d6-6a65-4a36-ba6e-57f08c96ce3f None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:49:09.549040 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-7870b0d6-6a65-4a36-ba6e-57f08c96ce3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:49:09.614199 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 574/1175] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 10 11:49:09 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 10 11:49:09.883981 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4879f0c-05cd-4d13-bbd0-795dcb3ba5dd None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:49:09.981676 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4879f0c-05cd-4d13-bbd0-795dcb3ba5dd None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:49:10.015093 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84cf0cf-0373-40bf-9a75-37e62e392186 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:49:10.015721 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84cf0cf-0373-40bf-9a75-37e62e392186 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:49:10.016005 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84cf0cf-0373-40bf-9a75-37e62e392186 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:49:10.016281 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84cf0cf-0373-40bf-9a75-37e62e392186 None admin] PATH_INFO: `/v3/domains/bddd0eaba1f44112baa754610413b793` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:49:10.017400 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84cf0cf-0373-40bf-9a75-37e62e392186 None admin] RBAC: Authorizing `identity:get_domain(domain_id=bddd0eaba1f44112baa754610413b793)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:49:10.018851 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84cf0cf-0373-40bf-9a75-37e62e392186 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:49:10.020419 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 602/1176] 10.222.0.46 () {56 vars in 1183 bytes} [Mon Oct 10 11:49:09 2022] GET /identity/v3/domains/bddd0eaba1f44112baa754610413b793 => generated 267 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:49:10.026247 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7870b0d6-6a65-4a36-ba6e-57f08c96ce3f None None] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:49:10.123873 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7870b0d6-6a65-4a36-ba6e-57f08c96ce3f None None] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:49:10.146542 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:49:10.157453 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d0854a-3f7e-4ccf-8bc2-e1416be8f4cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:49:10.158066 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d0854a-3f7e-4ccf-8bc2-e1416be8f4cd None admin] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:49:10.158349 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d0854a-3f7e-4ccf-8bc2-e1416be8f4cd None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:49:10.158626 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d0854a-3f7e-4ccf-8bc2-e1416be8f4cd None admin] PATH_INFO: `/v3/users` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:49:10.159869 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d0854a-3f7e-4ccf-8bc2-e1416be8f4cd None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:49:10.160769 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d0854a-3f7e-4ccf-8bc2-e1416be8f4cd None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:49:10.246761 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 575/1177] 10.222.0.46 () {60 vars in 1132 bytes} [Mon Oct 10 11:49:10 2022] POST /identity/v3/users => generated 311 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 11:49:12.450232 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccd94e9-c565-43c9-bf39-ab7bf8113b37 None None] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:49:12.450810 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccd94e9-c565-43c9-bf39-ab7bf8113b37 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:49:12.450810 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccd94e9-c565-43c9-bf39-ab7bf8113b37 None None] PATH_INFO: `/` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:49:12.452145 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 603/1178] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 10 11:49:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:49:12.457424 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f78fd65-e860-4cad-91ee-61f3489c0aef None None] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:49:12.472477 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f78fd65-e860-4cad-91ee-61f3489c0aef None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:49:12.472477 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f78fd65-e860-4cad-91ee-61f3489c0aef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:49:12.477245 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.auth.core [None req-6f78fd65-e860-4cad-91ee-61f3489c0aef None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:49:12.478452 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-6f78fd65-e860-4cad-91ee-61f3489c0aef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:49:12.546031 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 576/1179] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 10 11:49:12 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 10 11:49:12.815985 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccd94e9-c565-43c9-bf39-ab7bf8113b37 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:49:12.862751 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:49:12.914518 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccd94e9-c565-43c9-bf39-ab7bf8113b37 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:49:12.947224 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8100be6-dc63-4a64-a9fb-cdc7710ac618 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:49:12.947847 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8100be6-dc63-4a64-a9fb-cdc7710ac618 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:49:12.948150 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8100be6-dc63-4a64-a9fb-cdc7710ac618 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:49:12.948420 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8100be6-dc63-4a64-a9fb-cdc7710ac618 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:49:12.958247 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8100be6-dc63-4a64-a9fb-cdc7710ac618 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 10 11:49:12.963077 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application [None req-a8100be6-dc63-4a64-a9fb-cdc7710ac618 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 10 11:49:12.963077 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:49:12.963077 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:49:12.963077 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:49:12.963077 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:49:12.963077 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:49:12.963077 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:49:12.963077 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:49:12.963077 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:49:12.963077 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:49:12.963077 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:49:12.963077 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:49:12.963077 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 10 11:49:12.963077 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 10 11:49:12.963077 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 10 11:49:12.963077 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 10 11:49:12.964168 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 10 11:49:12.964168 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 10 11:49:12.964168 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 10 11:49:12.964168 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 10 11:49:12.964168 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:49:12.964168 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:49:12.964168 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:49:12.964168 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:49:12.964168 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:49:12.964168 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:49:12.964168 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:49:12.964168 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application with Lock( Oct 10 11:49:12.964168 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:49:12.964168 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:49:12.964168 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:49:12.964168 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:49:12.965341 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:49:12.965341 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:49:12.965341 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:49:12.965341 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:49:12.965341 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 10 11:49:12.965341 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 10 11:49:12.965341 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 10 11:49:12.965341 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 10 11:49:12.965341 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 10 11:49:12.965341 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 10 11:49:12.965341 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 10 11:49:12.965341 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Oct 10 11:49:12.965341 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 604/1180] 10.222.0.46 () {56 vars in 1096 bytes} [Mon Oct 10 11:49:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:49:12.970236 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f78fd65-e860-4cad-91ee-61f3489c0aef None None] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:49:13.066653 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f78fd65-e860-4cad-91ee-61f3489c0aef None None] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:49:13.077576 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:49:13.101019 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b348643-06b3-4f55-8cee-fe95401379ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:49:13.101635 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b348643-06b3-4f55-8cee-fe95401379ba None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:49:13.101919 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b348643-06b3-4f55-8cee-fe95401379ba None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:49:13.102194 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b348643-06b3-4f55-8cee-fe95401379ba None admin] PATH_INFO: `/v3/roles` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:49:13.102895 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b348643-06b3-4f55-8cee-fe95401379ba None admin] RBAC: Adding query filter params (name=admin) {{(pid=81676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:49:13.103203 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b348643-06b3-4f55-8cee-fe95401379ba None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:49:13.104565 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b348643-06b3-4f55-8cee-fe95401379ba None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:49:13.112491 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 577/1181] 10.222.0.46 () {56 vars in 1099 bytes} [Mon Oct 10 11:49:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:49:13.119464 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8100be6-dc63-4a64-a9fb-cdc7710ac618 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:49:13.217288 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8100be6-dc63-4a64-a9fb-cdc7710ac618 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:49:13.224453 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:49:13.250958 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0253337f-9498-4daf-b684-113bd6e14eca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:49:13.251826 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0253337f-9498-4daf-b684-113bd6e14eca None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:49:13.252285 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0253337f-9498-4daf-b684-113bd6e14eca None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:49:13.252859 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0253337f-9498-4daf-b684-113bd6e14eca None admin] PATH_INFO: `/v3/users/e00a67fcb84a46e590981be1d00d6161` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:49:13.265691 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0253337f-9498-4daf-b684-113bd6e14eca None admin] RBAC: Authorizing `identity:get_user(user_id=e00a67fcb84a46e590981be1d00d6161)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:49:13.267148 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0253337f-9498-4daf-b684-113bd6e14eca None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:49:13.268755 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 605/1182] 10.222.0.46 () {56 vars in 1177 bytes} [Mon Oct 10 11:49:13 2022] GET /identity/v3/users/e00a67fcb84a46e590981be1d00d6161 => generated 311 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:49:13.275202 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b348643-06b3-4f55-8cee-fe95401379ba None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:49:13.374169 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b348643-06b3-4f55-8cee-fe95401379ba None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:49:13.408361 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adad26ce-fd70-424e-90bc-73a19296ce5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:49:13.408987 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adad26ce-fd70-424e-90bc-73a19296ce5c None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:49:13.409263 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adad26ce-fd70-424e-90bc-73a19296ce5c None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:49:13.409533 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adad26ce-fd70-424e-90bc-73a19296ce5c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:49:13.453224 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:49:13.479341 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adad26ce-fd70-424e-90bc-73a19296ce5c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:49:13.480788 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adad26ce-fd70-424e-90bc-73a19296ce5c None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:49:13.579042 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 578/1183] 10.222.0.46 () {58 vars in 1282 bytes} [Mon Oct 10 11:49:13 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 10 11:49:13.585055 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0253337f-9498-4daf-b684-113bd6e14eca None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:49:13.689883 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0253337f-9498-4daf-b684-113bd6e14eca None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:49:13.722536 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0cf4d6-ec98-4a74-b2e5-df12ad67e14e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:49:13.723184 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0cf4d6-ec98-4a74-b2e5-df12ad67e14e None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:49:13.723462 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0cf4d6-ec98-4a74-b2e5-df12ad67e14e None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:49:13.723734 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0cf4d6-ec98-4a74-b2e5-df12ad67e14e None admin] PATH_INFO: `/v3/projects/cdad6f3108c043c08299af6af0fbcbbc` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:49:13.724827 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0cf4d6-ec98-4a74-b2e5-df12ad67e14e None admin] RBAC: Authorizing `identity:get_project(project_id=cdad6f3108c043c08299af6af0fbcbbc)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:49:13.726745 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0cf4d6-ec98-4a74-b2e5-df12ad67e14e None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:49:13.728345 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 606/1184] 10.222.0.46 () {56 vars in 1186 bytes} [Mon Oct 10 11:49:13 2022] GET /identity/v3/projects/cdad6f3108c043c08299af6af0fbcbbc => generated 361 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:49:13.734439 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adad26ce-fd70-424e-90bc-73a19296ce5c None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:49:13.801372 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:49:13.830502 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adad26ce-fd70-424e-90bc-73a19296ce5c None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:49:13.863402 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337b9971-532e-4f93-8804-14397d77fdd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:49:13.863985 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337b9971-532e-4f93-8804-14397d77fdd7 None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:49:13.864261 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337b9971-532e-4f93-8804-14397d77fdd7 None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:49:13.864531 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337b9971-532e-4f93-8804-14397d77fdd7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:49:13.865246 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337b9971-532e-4f93-8804-14397d77fdd7 None admin] RBAC: Adding query filter params (role.id=0d822a0674b6495fac623877a3b38025, scope.project.id=cdad6f3108c043c08299af6af0fbcbbc, user.id=e00a67fcb84a46e590981be1d00d6161) {{(pid=81676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:49:13.865548 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337b9971-532e-4f93-8804-14397d77fdd7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:49:13.867281 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337b9971-532e-4f93-8804-14397d77fdd7 None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:49:13.872470 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 579/1185] 10.222.0.46 () {56 vars in 1374 bytes} [Mon Oct 10 11:49:13 2022] GET /identity/v3/role_assignments?user.id=e00a67fcb84a46e590981be1d00d6161&scope.project.id=cdad6f3108c043c08299af6af0fbcbbc&role.id=0d822a0674b6495fac623877a3b38025 => generated 259 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:49:16.009429 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d68fde-334d-4a69-8dd7-c43acf8d7e00 None None] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:49:16.009980 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d68fde-334d-4a69-8dd7-c43acf8d7e00 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:49:16.009980 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d68fde-334d-4a69-8dd7-c43acf8d7e00 None None] PATH_INFO: `/` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:49:16.011335 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 607/1186] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 10 11:49:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:49:16.016644 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8f386b-6798-47ba-a35c-697eb3a86e8b None None] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:49:16.017069 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8f386b-6798-47ba-a35c-697eb3a86e8b None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:49:16.017201 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8f386b-6798-47ba-a35c-697eb3a86e8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:49:16.035638 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.auth.core [None req-fe8f386b-6798-47ba-a35c-697eb3a86e8b None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:49:16.036869 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-fe8f386b-6798-47ba-a35c-697eb3a86e8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:49:16.100674 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 580/1187] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 10 11:49:16 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 10 11:49:16.363603 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d68fde-334d-4a69-8dd7-c43acf8d7e00 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:49:16.394979 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:49:16.465850 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d68fde-334d-4a69-8dd7-c43acf8d7e00 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:49:16.501031 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39aaea15-9588-46a4-8d6e-fc631c0d4d06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:49:16.501666 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39aaea15-9588-46a4-8d6e-fc631c0d4d06 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:49:16.501949 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39aaea15-9588-46a4-8d6e-fc631c0d4d06 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:49:16.502224 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39aaea15-9588-46a4-8d6e-fc631c0d4d06 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:49:16.512224 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39aaea15-9588-46a4-8d6e-fc631c0d4d06 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 10 11:49:16.517063 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application [None req-39aaea15-9588-46a4-8d6e-fc631c0d4d06 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 10 11:49:16.517063 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:49:16.517063 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:49:16.517063 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:49:16.517063 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:49:16.517063 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:49:16.517063 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:49:16.517063 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:49:16.517063 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:49:16.517063 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:49:16.517063 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:49:16.517063 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:49:16.517063 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 10 11:49:16.517063 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 10 11:49:16.517063 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 10 11:49:16.517063 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 10 11:49:16.518037 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 10 11:49:16.518037 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 10 11:49:16.518037 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 10 11:49:16.518037 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 10 11:49:16.518037 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:49:16.518037 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:49:16.518037 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:49:16.518037 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:49:16.518037 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:49:16.518037 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:49:16.518037 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:49:16.518037 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application with Lock( Oct 10 11:49:16.518037 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:49:16.518037 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:49:16.518037 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:49:16.518037 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:49:16.518968 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:49:16.518968 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:49:16.518968 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:49:16.518968 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:49:16.518968 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 10 11:49:16.518968 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 10 11:49:16.518968 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 10 11:49:16.518968 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 10 11:49:16.518968 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 10 11:49:16.518968 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 10 11:49:16.518968 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 10 11:49:16.518968 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Oct 10 11:49:16.518968 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 608/1188] 10.222.0.46 () {56 vars in 1096 bytes} [Mon Oct 10 11:49:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:49:16.524296 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8f386b-6798-47ba-a35c-697eb3a86e8b None None] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:49:16.588535 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:49:16.622610 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8f386b-6798-47ba-a35c-697eb3a86e8b None None] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:49:16.655859 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b093c8c9-bdf3-4372-8ed1-b1bf9266f400 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:49:16.656466 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b093c8c9-bdf3-4372-8ed1-b1bf9266f400 None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:49:16.656747 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b093c8c9-bdf3-4372-8ed1-b1bf9266f400 None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:49:16.657022 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b093c8c9-bdf3-4372-8ed1-b1bf9266f400 None admin] PATH_INFO: `/v3/roles` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:49:16.657686 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b093c8c9-bdf3-4372-8ed1-b1bf9266f400 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:49:16.657990 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b093c8c9-bdf3-4372-8ed1-b1bf9266f400 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:49:16.659386 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b093c8c9-bdf3-4372-8ed1-b1bf9266f400 None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:49:16.667822 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 581/1189] 10.222.0.46 () {56 vars in 1099 bytes} [Mon Oct 10 11:49:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 11:49:16.673805 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39aaea15-9588-46a4-8d6e-fc631c0d4d06 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:49:16.762288 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:49:16.770566 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39aaea15-9588-46a4-8d6e-fc631c0d4d06 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:49:16.804472 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24fc2574-ac01-44f5-a7b0-9023cf287da8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:49:16.805132 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fc2574-ac01-44f5-a7b0-9023cf287da8 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:49:16.805412 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fc2574-ac01-44f5-a7b0-9023cf287da8 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:49:16.805684 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fc2574-ac01-44f5-a7b0-9023cf287da8 None admin] PATH_INFO: `/v3/users/e00a67fcb84a46e590981be1d00d6161` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:49:16.806916 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24fc2574-ac01-44f5-a7b0-9023cf287da8 None admin] RBAC: Authorizing `identity:get_user(user_id=e00a67fcb84a46e590981be1d00d6161)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:49:16.808399 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24fc2574-ac01-44f5-a7b0-9023cf287da8 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:49:16.810103 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 609/1190] 10.222.0.46 () {56 vars in 1177 bytes} [Mon Oct 10 11:49:16 2022] GET /identity/v3/users/e00a67fcb84a46e590981be1d00d6161 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:49:16.816199 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b093c8c9-bdf3-4372-8ed1-b1bf9266f400 None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:49:16.915895 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b093c8c9-bdf3-4372-8ed1-b1bf9266f400 None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:49:16.942783 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:49:16.951133 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05ef203-c83e-4b2e-89c3-da89f74b19eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:49:16.951772 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05ef203-c83e-4b2e-89c3-da89f74b19eb None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:49:16.957705 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05ef203-c83e-4b2e-89c3-da89f74b19eb None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:49:16.957705 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05ef203-c83e-4b2e-89c3-da89f74b19eb None admin] PATH_INFO: `/v3/projects/cdad6f3108c043c08299af6af0fbcbbc` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:49:16.957705 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05ef203-c83e-4b2e-89c3-da89f74b19eb None admin] RBAC: Authorizing `identity:get_project(project_id=cdad6f3108c043c08299af6af0fbcbbc)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:49:16.957705 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05ef203-c83e-4b2e-89c3-da89f74b19eb None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:49:16.957705 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 582/1191] 10.222.0.46 () {56 vars in 1186 bytes} [Mon Oct 10 11:49:16 2022] GET /identity/v3/projects/cdad6f3108c043c08299af6af0fbcbbc => generated 361 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:49:16.962474 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24fc2574-ac01-44f5-a7b0-9023cf287da8 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:49:17.062279 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24fc2574-ac01-44f5-a7b0-9023cf287da8 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:49:17.095325 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93d3f0e-4fc2-4b18-96c1-88c22a5c66d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:49:17.095978 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93d3f0e-4fc2-4b18-96c1-88c22a5c66d2 None admin] REQUEST_METHOD: `PUT` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:49:17.096262 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93d3f0e-4fc2-4b18-96c1-88c22a5c66d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:49:17.096534 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93d3f0e-4fc2-4b18-96c1-88c22a5c66d2 None admin] PATH_INFO: `/v3/projects/cdad6f3108c043c08299af6af0fbcbbc/users/e00a67fcb84a46e590981be1d00d6161/roles/0d822a0674b6495fac623877a3b38025` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:49:17.098424 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93d3f0e-4fc2-4b18-96c1-88c22a5c66d2 None admin] RBAC: Authorizing `identity:create_grant(project_id=cdad6f3108c043c08299af6af0fbcbbc, user_id=e00a67fcb84a46e590981be1d00d6161, role_id=0d822a0674b6495fac623877a3b38025)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:49:17.099894 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93d3f0e-4fc2-4b18-96c1-88c22a5c66d2 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:49:17.171502 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 610/1192] 10.222.0.46 () {58 vars in 1439 bytes} [Mon Oct 10 11:49:16 2022] PUT /identity/v3/projects/cdad6f3108c043c08299af6af0fbcbbc/users/e00a67fcb84a46e590981be1d00d6161/roles/0d822a0674b6495fac623877a3b38025 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 11:49:19.282451 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ee3a10-bb5a-4b30-b8c5-1b7a6605a899 None None] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:49:19.283030 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ee3a10-bb5a-4b30-b8c5-1b7a6605a899 None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:49:19.283030 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ee3a10-bb5a-4b30-b8c5-1b7a6605a899 None None] PATH_INFO: `/` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:49:19.284353 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 583/1193] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 10 11:49:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:49:19.289552 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611f8ba1-a94f-4e1d-b551-8bdaaffb650f None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:49:19.289945 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611f8ba1-a94f-4e1d-b551-8bdaaffb650f None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:49:19.290093 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611f8ba1-a94f-4e1d-b551-8bdaaffb650f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:49:19.308125 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-611f8ba1-a94f-4e1d-b551-8bdaaffb650f None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:49:19.309315 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-611f8ba1-a94f-4e1d-b551-8bdaaffb650f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:49:19.323336 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:49:19.372339 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 611/1194] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 10 11:49:19 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 10 11:49:19.640634 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ee3a10-bb5a-4b30-b8c5-1b7a6605a899 None None] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:49:19.739340 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ee3a10-bb5a-4b30-b8c5-1b7a6605a899 None None] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:49:19.772309 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ef429c-fba9-4732-b9dd-f76ce9ef379a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:49:19.772933 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ef429c-fba9-4732-b9dd-f76ce9ef379a None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:49:19.773213 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ef429c-fba9-4732-b9dd-f76ce9ef379a None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:49:19.773486 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ef429c-fba9-4732-b9dd-f76ce9ef379a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:49:19.783171 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ef429c-fba9-4732-b9dd-f76ce9ef379a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 10 11:49:19.788510 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application [None req-22ef429c-fba9-4732-b9dd-f76ce9ef379a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 10 11:49:19.788510 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:49:19.788510 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:49:19.788510 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:49:19.788510 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:49:19.788510 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:49:19.788510 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:49:19.788510 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:49:19.788510 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:49:19.788510 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:49:19.788510 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:49:19.788510 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:49:19.788510 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 10 11:49:19.788510 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 10 11:49:19.788510 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 10 11:49:19.788510 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 10 11:49:19.789607 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 10 11:49:19.789607 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 10 11:49:19.789607 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 10 11:49:19.789607 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 10 11:49:19.789607 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:49:19.789607 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:49:19.789607 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:49:19.789607 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:49:19.789607 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:49:19.789607 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:49:19.789607 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:49:19.789607 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application with Lock( Oct 10 11:49:19.789607 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:49:19.789607 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:49:19.789607 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:49:19.789607 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:49:19.790440 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:49:19.790440 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:49:19.790440 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:49:19.790440 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:49:19.790440 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 10 11:49:19.790440 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 10 11:49:19.790440 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 10 11:49:19.790440 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 10 11:49:19.790440 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 10 11:49:19.790440 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 10 11:49:19.790440 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 10 11:49:19.790440 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Oct 10 11:49:19.790440 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 584/1195] 10.222.0.46 () {56 vars in 1096 bytes} [Mon Oct 10 11:49:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:49:19.795665 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611f8ba1-a94f-4e1d-b551-8bdaaffb650f None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:49:19.889333 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611f8ba1-a94f-4e1d-b551-8bdaaffb650f None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:49:19.922637 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac111e27-9fd5-4ce5-8f54-41421770dfae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:49:19.923208 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac111e27-9fd5-4ce5-8f54-41421770dfae None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:49:19.923492 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac111e27-9fd5-4ce5-8f54-41421770dfae None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:49:19.923767 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac111e27-9fd5-4ce5-8f54-41421770dfae None admin] PATH_INFO: `/v3/roles` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:49:19.924429 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac111e27-9fd5-4ce5-8f54-41421770dfae None admin] RBAC: Adding query filter params (name=admin) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:49:19.924733 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac111e27-9fd5-4ce5-8f54-41421770dfae None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:49:19.926088 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac111e27-9fd5-4ce5-8f54-41421770dfae None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:49:19.934149 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 612/1196] 10.222.0.46 () {56 vars in 1099 bytes} [Mon Oct 10 11:49:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:49:19.940140 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ef429c-fba9-4732-b9dd-f76ce9ef379a None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:49:20.000266 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:49:20.043530 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ef429c-fba9-4732-b9dd-f76ce9ef379a None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:49:20.076203 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4a9e91-94a4-4ead-9c67-50535bec4933 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:49:20.076831 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4a9e91-94a4-4ead-9c67-50535bec4933 None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:49:20.077112 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4a9e91-94a4-4ead-9c67-50535bec4933 None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:49:20.077385 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4a9e91-94a4-4ead-9c67-50535bec4933 None admin] PATH_INFO: `/v3/users/e00a67fcb84a46e590981be1d00d6161` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:49:20.078503 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4a9e91-94a4-4ead-9c67-50535bec4933 None admin] RBAC: Authorizing `identity:get_user(user_id=e00a67fcb84a46e590981be1d00d6161)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:49:20.079949 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4a9e91-94a4-4ead-9c67-50535bec4933 None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:49:20.081566 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 585/1197] 10.222.0.46 () {56 vars in 1177 bytes} [Mon Oct 10 11:49:19 2022] GET /identity/v3/users/e00a67fcb84a46e590981be1d00d6161 => generated 311 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:49:20.087916 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac111e27-9fd5-4ce5-8f54-41421770dfae None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:49:20.104328 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:49:20.187267 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac111e27-9fd5-4ce5-8f54-41421770dfae None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:49:20.220154 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e53caa-7dc1-4493-8805-e531c0983625 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:49:20.220761 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e53caa-7dc1-4493-8805-e531c0983625 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:49:20.221040 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e53caa-7dc1-4493-8805-e531c0983625 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:49:20.221313 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e53caa-7dc1-4493-8805-e531c0983625 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:49:20.285389 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e53caa-7dc1-4493-8805-e531c0983625 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:49:20.286924 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e53caa-7dc1-4493-8805-e531c0983625 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:49:20.300154 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:49:20.382074 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 613/1198] 10.222.0.46 () {58 vars in 1282 bytes} [Mon Oct 10 11:49:20 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 10 11:49:20.388072 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4a9e91-94a4-4ead-9c67-50535bec4933 None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:49:20.488000 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4a9e91-94a4-4ead-9c67-50535bec4933 None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:49:20.511516 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:49:20.521738 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfd8b36-f2c7-4f3d-8ee7-29a797acb325 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:49:20.526936 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfd8b36-f2c7-4f3d-8ee7-29a797acb325 None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:49:20.526936 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfd8b36-f2c7-4f3d-8ee7-29a797acb325 None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:49:20.526936 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfd8b36-f2c7-4f3d-8ee7-29a797acb325 None admin] PATH_INFO: `/v3/projects/cdad6f3108c043c08299af6af0fbcbbc` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:49:20.526936 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcfd8b36-f2c7-4f3d-8ee7-29a797acb325 None admin] RBAC: Authorizing `identity:get_project(project_id=cdad6f3108c043c08299af6af0fbcbbc)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:49:20.526936 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcfd8b36-f2c7-4f3d-8ee7-29a797acb325 None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:49:20.527619 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 586/1199] 10.222.0.46 () {56 vars in 1186 bytes} [Mon Oct 10 11:49:20 2022] GET /identity/v3/projects/cdad6f3108c043c08299af6af0fbcbbc => generated 361 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:49:20.537157 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e53caa-7dc1-4493-8805-e531c0983625 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:49:20.638391 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e53caa-7dc1-4493-8805-e531c0983625 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:49:20.651188 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:49:20.671611 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c53af3-e76c-4860-aa00-e97e372da6a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:49:20.672213 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c53af3-e76c-4860-aa00-e97e372da6a7 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:49:20.672495 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c53af3-e76c-4860-aa00-e97e372da6a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:49:20.672877 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c53af3-e76c-4860-aa00-e97e372da6a7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:49:20.673605 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c53af3-e76c-4860-aa00-e97e372da6a7 None admin] RBAC: Adding query filter params (role.id=0d822a0674b6495fac623877a3b38025, scope.project.id=cdad6f3108c043c08299af6af0fbcbbc, user.id=e00a67fcb84a46e590981be1d00d6161) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:49:20.673916 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c53af3-e76c-4860-aa00-e97e372da6a7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:49:20.675384 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c53af3-e76c-4860-aa00-e97e372da6a7 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:49:20.681680 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 614/1200] 10.222.0.46 () {56 vars in 1374 bytes} [Mon Oct 10 11:49:20 2022] GET /identity/v3/role_assignments?user.id=e00a67fcb84a46e590981be1d00d6161&scope.project.id=cdad6f3108c043c08299af6af0fbcbbc&role.id=0d822a0674b6495fac623877a3b38025 => generated 608 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:49:30.543748 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=4 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:49:30.689756 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=12 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:50:38.731259 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447e2107-054b-4d35-8e87-dc4b5342ba64 None None] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:50:38.731922 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447e2107-054b-4d35-8e87-dc4b5342ba64 None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:50:38.731922 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447e2107-054b-4d35-8e87-dc4b5342ba64 None None] PATH_INFO: `/` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:50:38.733134 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 587/1201] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 10 11:50:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:50:38.739193 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08409c5d-828a-459c-bdd3-18151751f7b8 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:50:38.739616 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08409c5d-828a-459c-bdd3-18151751f7b8 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:50:38.739825 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08409c5d-828a-459c-bdd3-18151751f7b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:50:38.758352 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-08409c5d-828a-459c-bdd3-18151751f7b8 None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:50:38.759661 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-08409c5d-828a-459c-bdd3-18151751f7b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:50:38.825855 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 615/1202] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 10 11:50:38 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 10 11:50:39.100937 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447e2107-054b-4d35-8e87-dc4b5342ba64 None None] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:50:39.203878 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447e2107-054b-4d35-8e87-dc4b5342ba64 None None] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:50:39.241585 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e28973-eba0-49e0-aedf-28f231dc0acc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:50:39.242221 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e28973-eba0-49e0-aedf-28f231dc0acc None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:50:39.242500 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e28973-eba0-49e0-aedf-28f231dc0acc None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:50:39.242805 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e28973-eba0-49e0-aedf-28f231dc0acc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:50:39.253470 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e28973-eba0-49e0-aedf-28f231dc0acc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:50:39.254919 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e28973-eba0-49e0-aedf-28f231dc0acc None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:50:39.270052 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application [None req-03e28973-eba0-49e0-aedf-28f231dc0acc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:50:39.270052 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:50:39.270052 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:50:39.270052 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:50:39.270052 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:50:39.270052 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:50:39.270052 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:50:39.270052 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:50:39.270052 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:50:39.270052 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:50:39.270052 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:50:39.270052 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:50:39.270052 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 10 11:50:39.270052 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 10 11:50:39.270052 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 10 11:50:39.270052 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 10 11:50:39.274847 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:50:39.274847 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:50:39.274847 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:50:39.274847 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:50:39.274847 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:50:39.274847 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:50:39.274847 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:50:39.274847 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application with Lock( Oct 10 11:50:39.274847 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:50:39.274847 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:50:39.274847 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:50:39.274847 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:50:39.274847 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:50:39.274847 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:50:39.274847 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:50:39.274847 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:50:39.274847 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 10 11:50:39.276103 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 10 11:50:39.276103 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:50:39.276103 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Oct 10 11:50:39.276103 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 588/1203] 10.222.0.46 () {56 vars in 1108 bytes} [Mon Oct 10 11:50:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:50:39.277438 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08409c5d-828a-459c-bdd3-18151751f7b8 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:50:39.377434 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08409c5d-828a-459c-bdd3-18151751f7b8 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:50:39.407070 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:50:39.413018 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2e8cbd-39e0-44d0-aedc-65492440c5c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:50:39.413633 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2e8cbd-39e0-44d0-aedc-65492440c5c1 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:50:39.413913 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2e8cbd-39e0-44d0-aedc-65492440c5c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:50:39.414190 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2e8cbd-39e0-44d0-aedc-65492440c5c1 None admin] PATH_INFO: `/v3/domains` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:50:39.414857 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2e8cbd-39e0-44d0-aedc-65492440c5c1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:50:39.415165 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2e8cbd-39e0-44d0-aedc-65492440c5c1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:50:39.416536 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2e8cbd-39e0-44d0-aedc-65492440c5c1 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:50:39.428484 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 616/1204] 10.222.0.46 () {56 vars in 1109 bytes} [Mon Oct 10 11:50:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:50:39.436612 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e28973-eba0-49e0-aedf-28f231dc0acc None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:50:39.491235 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:50:39.539220 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e28973-eba0-49e0-aedf-28f231dc0acc None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:50:39.573795 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0463977f-3a15-41b3-9032-79d560e957f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:50:39.574407 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0463977f-3a15-41b3-9032-79d560e957f8 None admin] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:50:39.574711 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0463977f-3a15-41b3-9032-79d560e957f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:50:39.574995 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0463977f-3a15-41b3-9032-79d560e957f8 None admin] PATH_INFO: `/v3/users` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:50:39.575650 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0463977f-3a15-41b3-9032-79d560e957f8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:50:39.577132 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0463977f-3a15-41b3-9032-79d560e957f8 None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:50:39.694307 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 589/1205] 10.222.0.46 () {60 vars in 1132 bytes} [Mon Oct 10 11:50:39 2022] POST /identity/v3/users => generated 251 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 11:50:41.866387 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc25b1d7-859f-49d7-8a3f-78429ecec02a None None] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:50:41.866975 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc25b1d7-859f-49d7-8a3f-78429ecec02a None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:50:41.866975 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc25b1d7-859f-49d7-8a3f-78429ecec02a None None] PATH_INFO: `/` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:50:41.868225 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 617/1206] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 10 11:50:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:50:41.873656 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85b3e1c-b32e-4ea5-bea5-e6d54254759b None None] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:50:41.874101 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85b3e1c-b32e-4ea5-bea5-e6d54254759b None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:50:41.874203 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85b3e1c-b32e-4ea5-bea5-e6d54254759b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:50:41.892716 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.auth.core [None req-f85b3e1c-b32e-4ea5-bea5-e6d54254759b None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:50:41.894483 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-f85b3e1c-b32e-4ea5-bea5-e6d54254759b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:50:41.965261 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 590/1207] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 10 11:50:41 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 10 11:50:42.234697 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc25b1d7-859f-49d7-8a3f-78429ecec02a None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:50:42.337113 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc25b1d7-859f-49d7-8a3f-78429ecec02a None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:50:42.371436 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59238b37-48f3-4340-87da-57169f51a010 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:50:42.372085 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59238b37-48f3-4340-87da-57169f51a010 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:50:42.372350 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59238b37-48f3-4340-87da-57169f51a010 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:50:42.372623 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59238b37-48f3-4340-87da-57169f51a010 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:50:42.383432 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59238b37-48f3-4340-87da-57169f51a010 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 10 11:50:42.388287 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application [None req-59238b37-48f3-4340-87da-57169f51a010 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 10 11:50:42.388287 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:50:42.388287 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:50:42.388287 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:50:42.388287 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:50:42.388287 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:50:42.388287 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:50:42.388287 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:50:42.388287 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:50:42.388287 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:50:42.388287 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:50:42.388287 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:50:42.388287 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 10 11:50:42.388287 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 10 11:50:42.388287 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 10 11:50:42.388287 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 10 11:50:42.389412 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 10 11:50:42.389412 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 10 11:50:42.389412 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 10 11:50:42.389412 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 10 11:50:42.389412 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:50:42.389412 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:50:42.389412 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:50:42.389412 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:50:42.389412 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:50:42.389412 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:50:42.389412 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:50:42.389412 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application with Lock( Oct 10 11:50:42.389412 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:50:42.389412 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:50:42.389412 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:50:42.389412 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:50:42.390313 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:50:42.390313 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:50:42.390313 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:50:42.390313 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:50:42.390313 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 10 11:50:42.390313 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 10 11:50:42.390313 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 10 11:50:42.390313 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 10 11:50:42.390313 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 10 11:50:42.390313 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 10 11:50:42.390313 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 10 11:50:42.390313 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Oct 10 11:50:42.390313 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 618/1208] 10.222.0.46 () {56 vars in 1102 bytes} [Mon Oct 10 11:50:42 2022] GET /identity/v3/roles/service => generated 85 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:50:42.395718 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85b3e1c-b32e-4ea5-bea5-e6d54254759b None None] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:50:42.438699 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:50:42.509233 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85b3e1c-b32e-4ea5-bea5-e6d54254759b None None] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:50:42.546358 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a06b7f2-50be-4590-b34d-53266d04241a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:50:42.547011 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a06b7f2-50be-4590-b34d-53266d04241a None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:50:42.547292 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a06b7f2-50be-4590-b34d-53266d04241a None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:50:42.547565 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a06b7f2-50be-4590-b34d-53266d04241a None admin] PATH_INFO: `/v3/roles` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:50:42.548251 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a06b7f2-50be-4590-b34d-53266d04241a None admin] RBAC: Adding query filter params (name=service) {{(pid=81676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:50:42.548554 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a06b7f2-50be-4590-b34d-53266d04241a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:50:42.550454 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a06b7f2-50be-4590-b34d-53266d04241a None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:50:42.559300 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 591/1209] 10.222.0.46 () {56 vars in 1103 bytes} [Mon Oct 10 11:50:42 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:50:42.566289 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59238b37-48f3-4340-87da-57169f51a010 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:50:42.617736 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:50:42.665316 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59238b37-48f3-4340-87da-57169f51a010 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:50:42.697897 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f83a21-d553-41d6-8daa-9de39d15e501 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:50:42.698517 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f83a21-d553-41d6-8daa-9de39d15e501 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:50:42.698826 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f83a21-d553-41d6-8daa-9de39d15e501 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:50:42.699104 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f83a21-d553-41d6-8daa-9de39d15e501 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:50:42.709366 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f83a21-d553-41d6-8daa-9de39d15e501 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:50:42.710821 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f83a21-d553-41d6-8daa-9de39d15e501 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:50:42.724672 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application [None req-f5f83a21-d553-41d6-8daa-9de39d15e501 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:50:42.724672 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:50:42.724672 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:50:42.724672 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:50:42.724672 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:50:42.724672 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:50:42.724672 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:50:42.724672 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:50:42.724672 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:50:42.724672 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:50:42.724672 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:50:42.724672 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:50:42.724672 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 10 11:50:42.724672 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 10 11:50:42.724672 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 10 11:50:42.724672 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 10 11:50:42.728231 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:50:42.728231 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:50:42.728231 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:50:42.728231 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:50:42.728231 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:50:42.728231 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:50:42.728231 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:50:42.728231 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application with Lock( Oct 10 11:50:42.728231 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:50:42.728231 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:50:42.728231 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:50:42.728231 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:50:42.728231 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:50:42.728231 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:50:42.728231 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:50:42.728231 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:50:42.728231 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 10 11:50:42.729299 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 10 11:50:42.729299 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:50:42.729299 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Oct 10 11:50:42.729299 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 619/1210] 10.222.0.46 () {56 vars in 1108 bytes} [Mon Oct 10 11:50:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:50:42.731777 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a06b7f2-50be-4590-b34d-53266d04241a None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:50:42.845093 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:50:42.848995 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a06b7f2-50be-4590-b34d-53266d04241a None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:50:42.888335 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9241e4f-5fb8-4ab5-b720-77690911a30b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:50:42.889034 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9241e4f-5fb8-4ab5-b720-77690911a30b None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:50:42.889229 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9241e4f-5fb8-4ab5-b720-77690911a30b None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:50:42.889505 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9241e4f-5fb8-4ab5-b720-77690911a30b None admin] PATH_INFO: `/v3/domains` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:50:42.890169 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9241e4f-5fb8-4ab5-b720-77690911a30b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:50:42.890565 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9241e4f-5fb8-4ab5-b720-77690911a30b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:50:42.892153 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9241e4f-5fb8-4ab5-b720-77690911a30b None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:50:42.904698 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 592/1211] 10.222.0.46 () {56 vars in 1109 bytes} [Mon Oct 10 11:50:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:50:42.911508 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f83a21-d553-41d6-8daa-9de39d15e501 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:50:43.030757 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f83a21-d553-41d6-8daa-9de39d15e501 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:50:43.032820 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:50:43.073513 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721b8f11-16c8-447c-a2ca-66f77c060745 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:50:43.074158 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721b8f11-16c8-447c-a2ca-66f77c060745 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:50:43.074442 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721b8f11-16c8-447c-a2ca-66f77c060745 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:50:43.074751 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721b8f11-16c8-447c-a2ca-66f77c060745 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:50:43.083631 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721b8f11-16c8-447c-a2ca-66f77c060745 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:50:43.085155 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721b8f11-16c8-447c-a2ca-66f77c060745 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:50:43.098942 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application [None req-721b8f11-16c8-447c-a2ca-66f77c060745 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 10 11:50:43.098942 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:50:43.098942 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:50:43.098942 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:50:43.098942 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:50:43.098942 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:50:43.098942 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:50:43.098942 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:50:43.098942 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:50:43.098942 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:50:43.098942 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:50:43.098942 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:50:43.098942 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 10 11:50:43.098942 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 10 11:50:43.098942 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 10 11:50:43.098942 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 10 11:50:43.100078 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:50:43.100078 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:50:43.100078 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 10 11:50:43.100078 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 10 11:50:43.100078 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 10 11:50:43.100078 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 10 11:50:43.100078 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:50:43.100078 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:50:43.100078 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:50:43.100078 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:50:43.100078 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:50:43.100078 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application with Lock( Oct 10 11:50:43.100078 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:50:43.100078 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:50:43.100078 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:50:43.100078 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:50:43.100078 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:50:43.100078 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:50:43.101161 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:50:43.101161 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:50:43.101161 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 10 11:50:43.101161 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 10 11:50:43.101161 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 10 11:50:43.101161 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 10 11:50:43.101161 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 10 11:50:43.101161 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 10 11:50:43.101161 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 10 11:50:43.101161 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Oct 10 11:50:43.101161 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 620/1212] 10.222.0.46 () {56 vars in 1108 bytes} [Mon Oct 10 11:50:42 2022] GET /identity/v3/users/placement => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:50:43.106238 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9241e4f-5fb8-4ab5-b720-77690911a30b None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:50:43.216283 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9241e4f-5fb8-4ab5-b720-77690911a30b None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:50:43.254812 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181ab4d8-49b5-4541-aba3-2698fdce36c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:50:43.255439 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181ab4d8-49b5-4541-aba3-2698fdce36c7 None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:50:43.255723 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181ab4d8-49b5-4541-aba3-2698fdce36c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:50:43.256002 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181ab4d8-49b5-4541-aba3-2698fdce36c7 None admin] PATH_INFO: `/v3/users` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:50:43.256752 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181ab4d8-49b5-4541-aba3-2698fdce36c7 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:50:43.257063 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181ab4d8-49b5-4541-aba3-2698fdce36c7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:50:43.258514 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181ab4d8-49b5-4541-aba3-2698fdce36c7 None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:50:43.279838 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 593/1213] 10.222.0.46 () {56 vars in 1143 bytes} [Mon Oct 10 11:50:43 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:50:43.286468 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721b8f11-16c8-447c-a2ca-66f77c060745 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:50:43.431178 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721b8f11-16c8-447c-a2ca-66f77c060745 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:50:43.478531 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68475ad4-d480-41f6-bf06-ba965270dac3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:50:43.479175 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68475ad4-d480-41f6-bf06-ba965270dac3 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:50:43.479454 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68475ad4-d480-41f6-bf06-ba965270dac3 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:50:43.479726 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68475ad4-d480-41f6-bf06-ba965270dac3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:50:43.514600 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:50:43.556068 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68475ad4-d480-41f6-bf06-ba965270dac3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:50:43.557548 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68475ad4-d480-41f6-bf06-ba965270dac3 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:50:43.660957 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 621/1214] 10.222.0.46 () {58 vars in 1282 bytes} [Mon Oct 10 11:50:43 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 10 11:50:43.666979 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181ab4d8-49b5-4541-aba3-2698fdce36c7 None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:50:43.686369 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:50:43.774741 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181ab4d8-49b5-4541-aba3-2698fdce36c7 None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:50:43.812758 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8adc10d-12eb-493c-a60a-2155cf04d8b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:50:43.813391 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8adc10d-12eb-493c-a60a-2155cf04d8b8 None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:50:43.813716 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8adc10d-12eb-493c-a60a-2155cf04d8b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:50:43.814018 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8adc10d-12eb-493c-a60a-2155cf04d8b8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:50:43.825818 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8adc10d-12eb-493c-a60a-2155cf04d8b8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:50:43.827805 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8adc10d-12eb-493c-a60a-2155cf04d8b8 None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:50:43.844106 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application [None req-a8adc10d-12eb-493c-a60a-2155cf04d8b8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:50:43.844106 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:50:43.844106 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:50:43.844106 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:50:43.844106 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:50:43.844106 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:50:43.844106 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:50:43.844106 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:50:43.844106 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:50:43.844106 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:50:43.844106 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:50:43.844106 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:50:43.844106 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 10 11:50:43.844106 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 10 11:50:43.844106 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 10 11:50:43.844106 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 10 11:50:43.845641 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:50:43.845641 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:50:43.845641 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:50:43.845641 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:50:43.845641 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:50:43.845641 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:50:43.845641 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:50:43.845641 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application with Lock( Oct 10 11:50:43.845641 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:50:43.845641 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:50:43.845641 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:50:43.845641 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:50:43.845641 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:50:43.845641 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:50:43.845641 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:50:43.845641 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:50:43.845641 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 10 11:50:43.846524 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 10 11:50:43.846524 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:50:43.846524 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Oct 10 11:50:43.846524 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 594/1215] 10.222.0.46 () {56 vars in 1108 bytes} [Mon Oct 10 11:50:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:50:43.851440 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68475ad4-d480-41f6-bf06-ba965270dac3 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:50:43.918281 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:50:43.956396 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68475ad4-d480-41f6-bf06-ba965270dac3 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:50:43.992078 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277a488b-b28e-4946-a7a4-0abbeb326d05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:50:43.992682 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277a488b-b28e-4946-a7a4-0abbeb326d05 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:50:43.992956 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277a488b-b28e-4946-a7a4-0abbeb326d05 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:50:43.993225 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277a488b-b28e-4946-a7a4-0abbeb326d05 None admin] PATH_INFO: `/v3/domains` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:50:43.993863 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277a488b-b28e-4946-a7a4-0abbeb326d05 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:50:43.994158 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277a488b-b28e-4946-a7a4-0abbeb326d05 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:50:43.995593 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277a488b-b28e-4946-a7a4-0abbeb326d05 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:50:44.008938 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 622/1216] 10.222.0.46 () {56 vars in 1109 bytes} [Mon Oct 10 11:50:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:50:44.014827 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8adc10d-12eb-493c-a60a-2155cf04d8b8 None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:50:44.090520 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:50:44.122378 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8adc10d-12eb-493c-a60a-2155cf04d8b8 None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:50:44.163267 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52afc699-1156-4400-95d3-080df068f9d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:50:44.163909 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52afc699-1156-4400-95d3-080df068f9d7 None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:50:44.164194 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52afc699-1156-4400-95d3-080df068f9d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:50:44.164470 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52afc699-1156-4400-95d3-080df068f9d7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:50:44.171989 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52afc699-1156-4400-95d3-080df068f9d7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:50:44.173571 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52afc699-1156-4400-95d3-080df068f9d7 None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:50:44.186242 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application [None req-52afc699-1156-4400-95d3-080df068f9d7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 10 11:50:44.186242 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:50:44.186242 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:50:44.186242 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:50:44.186242 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:50:44.186242 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:50:44.186242 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:50:44.186242 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:50:44.186242 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:50:44.186242 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:50:44.186242 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:50:44.186242 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:50:44.186242 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 10 11:50:44.186242 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 10 11:50:44.186242 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 10 11:50:44.187332 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 10 11:50:44.187332 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:50:44.187332 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:50:44.187332 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:50:44.187332 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:50:44.187332 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:50:44.187332 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:50:44.187332 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:50:44.187332 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application with Lock( Oct 10 11:50:44.187332 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:50:44.187332 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:50:44.187332 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:50:44.187332 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:50:44.187332 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:50:44.187332 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:50:44.187332 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:50:44.187332 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:50:44.188406 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 10 11:50:44.188406 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 10 11:50:44.188406 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 10 11:50:44.188406 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 10 11:50:44.188406 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 10 11:50:44.188406 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 10 11:50:44.188406 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 10 11:50:44.188406 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Oct 10 11:50:44.188406 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 595/1217] 10.222.0.46 () {56 vars in 1111 bytes} [Mon Oct 10 11:50:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:50:44.195187 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277a488b-b28e-4946-a7a4-0abbeb326d05 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:50:44.308393 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277a488b-b28e-4946-a7a4-0abbeb326d05 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:50:44.331561 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:50:44.342777 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5d43d4-c2be-4cd3-a156-f45c6d8ed366 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:50:44.343388 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5d43d4-c2be-4cd3-a156-f45c6d8ed366 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:50:44.343669 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5d43d4-c2be-4cd3-a156-f45c6d8ed366 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:50:44.343944 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5d43d4-c2be-4cd3-a156-f45c6d8ed366 None admin] PATH_INFO: `/v3/projects` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:50:44.344690 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5d43d4-c2be-4cd3-a156-f45c6d8ed366 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:50:44.344995 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5d43d4-c2be-4cd3-a156-f45c6d8ed366 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:50:44.346988 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5d43d4-c2be-4cd3-a156-f45c6d8ed366 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:50:44.358431 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 623/1218] 10.222.0.46 () {56 vars in 1148 bytes} [Mon Oct 10 11:50:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:50:44.365993 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52afc699-1156-4400-95d3-080df068f9d7 None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:50:44.469552 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52afc699-1156-4400-95d3-080df068f9d7 None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:50:44.498972 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:50:44.504871 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8679576-f44b-438a-aa88-3a50d1c51bf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:50:44.505487 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8679576-f44b-438a-aa88-3a50d1c51bf2 None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:50:44.505763 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8679576-f44b-438a-aa88-3a50d1c51bf2 None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:50:44.506039 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8679576-f44b-438a-aa88-3a50d1c51bf2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:50:44.506799 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8679576-f44b-438a-aa88-3a50d1c51bf2 None admin] RBAC: Adding query filter params (role.id=3c35be38787740f4834c728a2ce73b6b, scope.project.id=4ad3772f7b6a47c28b7ba37a9316a3c1, user.id=8e8f9bb1776d47169ef7febb57c954cf) {{(pid=81676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:50:44.507111 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8679576-f44b-438a-aa88-3a50d1c51bf2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:50:44.508533 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8679576-f44b-438a-aa88-3a50d1c51bf2 None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:50:44.513964 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 596/1219] 10.222.0.46 () {56 vars in 1374 bytes} [Mon Oct 10 11:50:44 2022] GET /identity/v3/role_assignments?user.id=8e8f9bb1776d47169ef7febb57c954cf&scope.project.id=4ad3772f7b6a47c28b7ba37a9316a3c1&role.id=3c35be38787740f4834c728a2ce73b6b => generated 259 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:50:46.683164 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d832a7-2b16-452f-8d97-78cac5f3b762 None None] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:50:46.683693 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d832a7-2b16-452f-8d97-78cac5f3b762 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:50:46.683780 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d832a7-2b16-452f-8d97-78cac5f3b762 None None] PATH_INFO: `/` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:50:46.685109 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 624/1220] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 10 11:50:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:50:46.691356 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7463eb-debb-4ceb-ae62-9b7409247d5e None None] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:50:46.691877 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7463eb-debb-4ceb-ae62-9b7409247d5e None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:50:46.691976 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7463eb-debb-4ceb-ae62-9b7409247d5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:50:46.710568 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.auth.core [None req-cf7463eb-debb-4ceb-ae62-9b7409247d5e None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:50:46.711789 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-cf7463eb-debb-4ceb-ae62-9b7409247d5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:50:46.779916 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 597/1221] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 10 11:50:46 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 10 11:50:47.056401 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d832a7-2b16-452f-8d97-78cac5f3b762 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:50:47.163049 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d832a7-2b16-452f-8d97-78cac5f3b762 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:50:47.200168 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7b4079-05df-40f0-8a5e-79e9aa0fd9c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:50:47.205352 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7b4079-05df-40f0-8a5e-79e9aa0fd9c2 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:50:47.205352 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7b4079-05df-40f0-8a5e-79e9aa0fd9c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:50:47.205352 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7b4079-05df-40f0-8a5e-79e9aa0fd9c2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:50:47.212881 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7b4079-05df-40f0-8a5e-79e9aa0fd9c2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 10 11:50:47.217945 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application [None req-fa7b4079-05df-40f0-8a5e-79e9aa0fd9c2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 10 11:50:47.217945 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:50:47.217945 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:50:47.217945 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:50:47.217945 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:50:47.217945 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:50:47.217945 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:50:47.217945 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:50:47.217945 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:50:47.217945 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:50:47.217945 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:50:47.217945 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:50:47.217945 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 10 11:50:47.217945 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 10 11:50:47.217945 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 10 11:50:47.217945 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 10 11:50:47.219181 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 10 11:50:47.219181 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 10 11:50:47.219181 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 10 11:50:47.219181 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 10 11:50:47.219181 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:50:47.219181 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:50:47.219181 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:50:47.219181 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:50:47.219181 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:50:47.219181 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:50:47.219181 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:50:47.219181 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application with Lock( Oct 10 11:50:47.219181 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:50:47.219181 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:50:47.219181 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:50:47.219181 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:50:47.220562 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:50:47.220562 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:50:47.220562 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:50:47.220562 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:50:47.220562 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 10 11:50:47.220562 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 10 11:50:47.220562 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 10 11:50:47.220562 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 10 11:50:47.220562 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 10 11:50:47.220562 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 10 11:50:47.220562 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 10 11:50:47.220562 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Oct 10 11:50:47.220562 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 625/1222] 10.222.0.46 () {56 vars in 1102 bytes} [Mon Oct 10 11:50:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:50:47.226031 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7463eb-debb-4ceb-ae62-9b7409247d5e None None] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:50:47.336519 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7463eb-debb-4ceb-ae62-9b7409247d5e None None] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:50:47.364815 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:50:47.376201 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe64b80-255b-427d-9134-3fa634229ca2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:50:47.376830 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe64b80-255b-427d-9134-3fa634229ca2 None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:50:47.377115 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe64b80-255b-427d-9134-3fa634229ca2 None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:50:47.377393 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe64b80-255b-427d-9134-3fa634229ca2 None admin] PATH_INFO: `/v3/roles` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:50:47.378129 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe64b80-255b-427d-9134-3fa634229ca2 None admin] RBAC: Adding query filter params (name=service) {{(pid=81676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:50:47.378436 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe64b80-255b-427d-9134-3fa634229ca2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:50:47.379897 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe64b80-255b-427d-9134-3fa634229ca2 None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:50:47.389481 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 598/1223] 10.222.0.46 () {56 vars in 1103 bytes} [Mon Oct 10 11:50:47 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:50:47.395880 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7b4079-05df-40f0-8a5e-79e9aa0fd9c2 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:50:47.445069 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:50:47.513436 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7b4079-05df-40f0-8a5e-79e9aa0fd9c2 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:50:47.548224 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c761be38-9431-49a8-aa90-7a4442afed65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:50:47.548859 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c761be38-9431-49a8-aa90-7a4442afed65 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:50:47.549142 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c761be38-9431-49a8-aa90-7a4442afed65 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:50:47.549416 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c761be38-9431-49a8-aa90-7a4442afed65 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:50:47.561664 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c761be38-9431-49a8-aa90-7a4442afed65 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:50:47.563622 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c761be38-9431-49a8-aa90-7a4442afed65 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:50:47.580510 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application [None req-c761be38-9431-49a8-aa90-7a4442afed65 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:50:47.580510 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:50:47.580510 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:50:47.580510 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:50:47.580510 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:50:47.580510 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:50:47.580510 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:50:47.580510 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:50:47.580510 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:50:47.580510 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:50:47.580510 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:50:47.580510 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:50:47.580510 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 10 11:50:47.580510 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 10 11:50:47.580510 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 10 11:50:47.580510 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 10 11:50:47.582131 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:50:47.582131 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:50:47.582131 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:50:47.582131 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:50:47.582131 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:50:47.582131 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:50:47.582131 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:50:47.582131 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application with Lock( Oct 10 11:50:47.582131 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:50:47.582131 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:50:47.582131 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:50:47.582131 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:50:47.582131 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:50:47.582131 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:50:47.582131 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:50:47.582131 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:50:47.582131 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 10 11:50:47.583173 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 10 11:50:47.583173 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:50:47.583173 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Oct 10 11:50:47.583173 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 626/1224] 10.222.0.46 () {56 vars in 1108 bytes} [Mon Oct 10 11:50:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 10 11:50:47.588115 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe64b80-255b-427d-9134-3fa634229ca2 None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:50:47.696224 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe64b80-255b-427d-9134-3fa634229ca2 None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:50:47.732829 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936a8868-91b4-4d48-bbf7-c15b2faaf6fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:50:47.733436 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936a8868-91b4-4d48-bbf7-c15b2faaf6fa None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:50:47.733715 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936a8868-91b4-4d48-bbf7-c15b2faaf6fa None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:50:47.733989 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936a8868-91b4-4d48-bbf7-c15b2faaf6fa None admin] PATH_INFO: `/v3/domains` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:50:47.734634 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936a8868-91b4-4d48-bbf7-c15b2faaf6fa None admin] RBAC: Adding query filter params (name=Default) {{(pid=81676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:50:47.734981 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936a8868-91b4-4d48-bbf7-c15b2faaf6fa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:50:47.739975 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936a8868-91b4-4d48-bbf7-c15b2faaf6fa None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:50:47.748573 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 599/1225] 10.222.0.46 () {56 vars in 1109 bytes} [Mon Oct 10 11:50:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 11:50:47.754775 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c761be38-9431-49a8-aa90-7a4442afed65 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:50:47.847352 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:50:47.854222 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c761be38-9431-49a8-aa90-7a4442afed65 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:50:47.887853 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9b5fe2-7aa3-438d-bcd6-6dca739717d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:50:47.888475 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9b5fe2-7aa3-438d-bcd6-6dca739717d7 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:50:47.888760 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9b5fe2-7aa3-438d-bcd6-6dca739717d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:50:47.889034 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9b5fe2-7aa3-438d-bcd6-6dca739717d7 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:50:47.896501 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9b5fe2-7aa3-438d-bcd6-6dca739717d7 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:50:47.897926 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9b5fe2-7aa3-438d-bcd6-6dca739717d7 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:50:47.910044 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application [None req-9d9b5fe2-7aa3-438d-bcd6-6dca739717d7 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 10 11:50:47.910044 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:50:47.910044 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:50:47.910044 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:50:47.910044 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:50:47.910044 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:50:47.910044 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:50:47.910044 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:50:47.910044 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:50:47.910044 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:50:47.910044 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:50:47.910044 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:50:47.910044 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 10 11:50:47.910044 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 10 11:50:47.910044 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 10 11:50:47.910044 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 10 11:50:47.911182 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:50:47.911182 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:50:47.911182 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 10 11:50:47.911182 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 10 11:50:47.911182 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 10 11:50:47.911182 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 10 11:50:47.911182 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:50:47.911182 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:50:47.911182 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:50:47.911182 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:50:47.911182 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:50:47.911182 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application with Lock( Oct 10 11:50:47.911182 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:50:47.911182 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:50:47.911182 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:50:47.911182 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:50:47.911182 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:50:47.911182 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:50:47.912567 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:50:47.912567 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:50:47.912567 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 10 11:50:47.912567 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 10 11:50:47.912567 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 10 11:50:47.912567 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 10 11:50:47.912567 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 10 11:50:47.912567 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 10 11:50:47.912567 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 10 11:50:47.912567 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Oct 10 11:50:47.912567 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 627/1226] 10.222.0.46 () {56 vars in 1108 bytes} [Mon Oct 10 11:50:47 2022] GET /identity/v3/users/placement => generated 87 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:50:47.917132 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936a8868-91b4-4d48-bbf7-c15b2faaf6fa None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:50:47.964598 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:50:48.018502 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936a8868-91b4-4d48-bbf7-c15b2faaf6fa None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:50:48.052136 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1154c5f0-cd88-4a77-8f20-82e0319bc401 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:50:48.052735 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1154c5f0-cd88-4a77-8f20-82e0319bc401 None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:50:48.053014 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1154c5f0-cd88-4a77-8f20-82e0319bc401 None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:50:48.053289 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1154c5f0-cd88-4a77-8f20-82e0319bc401 None admin] PATH_INFO: `/v3/users` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:50:48.054012 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1154c5f0-cd88-4a77-8f20-82e0319bc401 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:50:48.054390 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1154c5f0-cd88-4a77-8f20-82e0319bc401 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:50:48.055845 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1154c5f0-cd88-4a77-8f20-82e0319bc401 None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:50:48.073203 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 600/1227] 10.222.0.46 () {56 vars in 1143 bytes} [Mon Oct 10 11:50:47 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:50:48.079323 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9b5fe2-7aa3-438d-bcd6-6dca739717d7 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:50:48.179377 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9b5fe2-7aa3-438d-bcd6-6dca739717d7 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:50:48.214219 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40eb8fc4-1f43-42d3-9cb5-2da2450916c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:50:48.214901 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40eb8fc4-1f43-42d3-9cb5-2da2450916c4 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:50:48.215185 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40eb8fc4-1f43-42d3-9cb5-2da2450916c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:50:48.215458 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40eb8fc4-1f43-42d3-9cb5-2da2450916c4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:50:48.225912 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40eb8fc4-1f43-42d3-9cb5-2da2450916c4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:50:48.227453 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40eb8fc4-1f43-42d3-9cb5-2da2450916c4 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:50:48.241908 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application [None req-40eb8fc4-1f43-42d3-9cb5-2da2450916c4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:50:48.241908 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:50:48.241908 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:50:48.241908 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:50:48.241908 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:50:48.241908 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:50:48.241908 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:50:48.241908 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:50:48.241908 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:50:48.241908 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:50:48.241908 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:50:48.241908 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:50:48.241908 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 10 11:50:48.241908 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 10 11:50:48.241908 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 10 11:50:48.241908 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 10 11:50:48.243104 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:50:48.243104 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:50:48.243104 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:50:48.243104 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:50:48.243104 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:50:48.243104 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:50:48.243104 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:50:48.243104 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application with Lock( Oct 10 11:50:48.243104 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:50:48.243104 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:50:48.243104 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:50:48.243104 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:50:48.243104 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:50:48.243104 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:50:48.243104 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:50:48.243104 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:50:48.243104 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 10 11:50:48.244355 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 10 11:50:48.244355 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:50:48.244355 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Oct 10 11:50:48.244355 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 628/1228] 10.222.0.46 () {56 vars in 1108 bytes} [Mon Oct 10 11:50:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:50:48.249109 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1154c5f0-cd88-4a77-8f20-82e0319bc401 None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:50:48.349580 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1154c5f0-cd88-4a77-8f20-82e0319bc401 None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:50:48.351990 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:50:48.383881 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10c9ba0-3e55-4cc7-b6a4-bb3ff5ffe2eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:50:48.384483 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10c9ba0-3e55-4cc7-b6a4-bb3ff5ffe2eb None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:50:48.384765 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10c9ba0-3e55-4cc7-b6a4-bb3ff5ffe2eb None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:50:48.385041 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10c9ba0-3e55-4cc7-b6a4-bb3ff5ffe2eb None admin] PATH_INFO: `/v3/domains` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:50:48.385682 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10c9ba0-3e55-4cc7-b6a4-bb3ff5ffe2eb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:50:48.385990 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10c9ba0-3e55-4cc7-b6a4-bb3ff5ffe2eb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:50:48.387380 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10c9ba0-3e55-4cc7-b6a4-bb3ff5ffe2eb None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:50:48.402615 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 601/1229] 10.222.0.46 () {56 vars in 1109 bytes} [Mon Oct 10 11:50:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:50:48.408329 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40eb8fc4-1f43-42d3-9cb5-2da2450916c4 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:50:48.417781 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:50:48.519137 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40eb8fc4-1f43-42d3-9cb5-2da2450916c4 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:50:48.551611 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84c5088-7fdf-4a0c-8016-740f8fb43061 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:50:48.552234 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84c5088-7fdf-4a0c-8016-740f8fb43061 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:50:48.552513 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84c5088-7fdf-4a0c-8016-740f8fb43061 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:50:48.552779 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84c5088-7fdf-4a0c-8016-740f8fb43061 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:50:48.559229 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84c5088-7fdf-4a0c-8016-740f8fb43061 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:50:48.560680 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84c5088-7fdf-4a0c-8016-740f8fb43061 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:50:48.572382 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application [None req-a84c5088-7fdf-4a0c-8016-740f8fb43061 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 10 11:50:48.572382 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:50:48.572382 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:50:48.572382 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:50:48.572382 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:50:48.572382 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:50:48.572382 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:50:48.572382 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:50:48.572382 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:50:48.572382 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:50:48.572382 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:50:48.572382 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:50:48.572382 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 10 11:50:48.572382 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 10 11:50:48.572382 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 10 11:50:48.577530 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 10 11:50:48.577530 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:50:48.577530 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:50:48.577530 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:50:48.577530 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:50:48.577530 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:50:48.577530 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:50:48.577530 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:50:48.577530 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application with Lock( Oct 10 11:50:48.577530 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:50:48.577530 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:50:48.577530 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:50:48.577530 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:50:48.577530 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:50:48.577530 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:50:48.577530 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:50:48.577530 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:50:48.578420 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 10 11:50:48.578420 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 10 11:50:48.578420 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 10 11:50:48.578420 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 10 11:50:48.578420 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 10 11:50:48.578420 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 10 11:50:48.578420 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 10 11:50:48.578420 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Oct 10 11:50:48.578420 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 629/1230] 10.222.0.46 () {56 vars in 1111 bytes} [Mon Oct 10 11:50:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:50:48.579991 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10c9ba0-3e55-4cc7-b6a4-bb3ff5ffe2eb None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:50:48.683143 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10c9ba0-3e55-4cc7-b6a4-bb3ff5ffe2eb None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:50:48.716725 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36da7ff3-eae6-44b4-ad3e-35188dd2f652 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:50:48.717379 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36da7ff3-eae6-44b4-ad3e-35188dd2f652 None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:50:48.717636 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36da7ff3-eae6-44b4-ad3e-35188dd2f652 None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:50:48.717912 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36da7ff3-eae6-44b4-ad3e-35188dd2f652 None admin] PATH_INFO: `/v3/projects` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:50:48.718604 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36da7ff3-eae6-44b4-ad3e-35188dd2f652 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:50:48.718942 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36da7ff3-eae6-44b4-ad3e-35188dd2f652 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:50:48.720357 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36da7ff3-eae6-44b4-ad3e-35188dd2f652 None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:50:48.737801 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 602/1231] 10.222.0.46 () {56 vars in 1148 bytes} [Mon Oct 10 11:50:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 11:50:48.740171 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84c5088-7fdf-4a0c-8016-740f8fb43061 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:50:48.804084 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:50:48.837094 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84c5088-7fdf-4a0c-8016-740f8fb43061 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:50:48.870029 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce41d07-7bf6-42a0-bc5d-7afe15909e43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:50:48.870702 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce41d07-7bf6-42a0-bc5d-7afe15909e43 None admin] REQUEST_METHOD: `PUT` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:50:48.870993 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce41d07-7bf6-42a0-bc5d-7afe15909e43 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:50:48.871266 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce41d07-7bf6-42a0-bc5d-7afe15909e43 None admin] PATH_INFO: `/v3/projects/4ad3772f7b6a47c28b7ba37a9316a3c1/users/8e8f9bb1776d47169ef7febb57c954cf/roles/3c35be38787740f4834c728a2ce73b6b` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:50:48.885273 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce41d07-7bf6-42a0-bc5d-7afe15909e43 None admin] RBAC: Authorizing `identity:create_grant(project_id=4ad3772f7b6a47c28b7ba37a9316a3c1, user_id=8e8f9bb1776d47169ef7febb57c954cf, role_id=3c35be38787740f4834c728a2ce73b6b)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:50:48.888261 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce41d07-7bf6-42a0-bc5d-7afe15909e43 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:50:49.068129 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 630/1232] 10.222.0.46 () {58 vars in 1439 bytes} [Mon Oct 10 11:50:48 2022] PUT /identity/v3/projects/4ad3772f7b6a47c28b7ba37a9316a3c1/users/8e8f9bb1776d47169ef7febb57c954cf/roles/3c35be38787740f4834c728a2ce73b6b => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 11:50:51.213175 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050edfb3-e691-4cce-bea3-719e079896a9 None None] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:50:51.213717 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050edfb3-e691-4cce-bea3-719e079896a9 None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:50:51.214191 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050edfb3-e691-4cce-bea3-719e079896a9 None None] PATH_INFO: `/` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:50:51.216793 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 603/1233] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 10 11:50:51 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:50:51.223487 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ed94af-5cf4-4e8d-b61b-c6502c9edacd None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:50:51.223985 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ed94af-5cf4-4e8d-b61b-c6502c9edacd None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:50:51.224098 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ed94af-5cf4-4e8d-b61b-c6502c9edacd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:50:51.243535 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-c3ed94af-5cf4-4e8d-b61b-c6502c9edacd None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:50:51.244735 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-c3ed94af-5cf4-4e8d-b61b-c6502c9edacd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:50:51.317095 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 631/1234] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 10 11:50:51 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 10 11:50:51.589819 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050edfb3-e691-4cce-bea3-719e079896a9 None None] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:50:51.618265 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:50:51.701923 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050edfb3-e691-4cce-bea3-719e079896a9 None None] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:50:51.739397 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68401412-bb3a-4bce-908c-6937c6a55344 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:50:51.739947 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68401412-bb3a-4bce-908c-6937c6a55344 None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:50:51.740303 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68401412-bb3a-4bce-908c-6937c6a55344 None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:50:51.740703 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68401412-bb3a-4bce-908c-6937c6a55344 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:50:51.752106 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68401412-bb3a-4bce-908c-6937c6a55344 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 10 11:50:51.757468 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application [None req-68401412-bb3a-4bce-908c-6937c6a55344 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 10 11:50:51.757468 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:50:51.757468 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:50:51.757468 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:50:51.757468 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:50:51.757468 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:50:51.757468 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:50:51.757468 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:50:51.757468 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:50:51.757468 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:50:51.757468 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:50:51.757468 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:50:51.757468 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 10 11:50:51.757468 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 10 11:50:51.757468 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 10 11:50:51.757468 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 10 11:50:51.758909 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 10 11:50:51.758909 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 10 11:50:51.758909 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 10 11:50:51.758909 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 10 11:50:51.758909 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:50:51.758909 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:50:51.758909 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:50:51.758909 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:50:51.758909 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:50:51.758909 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:50:51.758909 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:50:51.758909 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application with Lock( Oct 10 11:50:51.758909 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:50:51.758909 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:50:51.758909 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:50:51.758909 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:50:51.759842 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:50:51.759842 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:50:51.759842 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:50:51.759842 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:50:51.759842 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 10 11:50:51.759842 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 10 11:50:51.759842 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 10 11:50:51.759842 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 10 11:50:51.759842 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 10 11:50:51.759842 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 10 11:50:51.759842 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 10 11:50:51.759842 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Oct 10 11:50:51.759842 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 604/1235] 10.222.0.46 () {56 vars in 1102 bytes} [Mon Oct 10 11:50:51 2022] GET /identity/v3/roles/service => generated 85 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 10 11:50:51.764755 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ed94af-5cf4-4e8d-b61b-c6502c9edacd None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:50:51.816556 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:50:51.876815 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ed94af-5cf4-4e8d-b61b-c6502c9edacd None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:50:51.916702 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b7ed20-3f0a-4505-9cb8-ca1ebc6b9b1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:50:51.917332 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b7ed20-3f0a-4505-9cb8-ca1ebc6b9b1b None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:50:51.917614 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b7ed20-3f0a-4505-9cb8-ca1ebc6b9b1b None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:50:51.917884 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b7ed20-3f0a-4505-9cb8-ca1ebc6b9b1b None admin] PATH_INFO: `/v3/roles` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:50:51.918571 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b7ed20-3f0a-4505-9cb8-ca1ebc6b9b1b None admin] RBAC: Adding query filter params (name=service) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:50:51.919001 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b7ed20-3f0a-4505-9cb8-ca1ebc6b9b1b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:50:51.920310 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b7ed20-3f0a-4505-9cb8-ca1ebc6b9b1b None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:50:51.929512 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 632/1236] 10.222.0.46 () {56 vars in 1103 bytes} [Mon Oct 10 11:50:51 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:50:51.938556 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68401412-bb3a-4bce-908c-6937c6a55344 None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:50:52.032406 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:50:52.052363 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68401412-bb3a-4bce-908c-6937c6a55344 None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:50:52.094467 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8978af7e-7758-45a5-9374-9eeb167f2245 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:50:52.095514 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8978af7e-7758-45a5-9374-9eeb167f2245 None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:50:52.095994 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8978af7e-7758-45a5-9374-9eeb167f2245 None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:50:52.096409 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8978af7e-7758-45a5-9374-9eeb167f2245 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:50:52.110658 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8978af7e-7758-45a5-9374-9eeb167f2245 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:50:52.112191 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8978af7e-7758-45a5-9374-9eeb167f2245 None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:50:52.137624 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application [None req-8978af7e-7758-45a5-9374-9eeb167f2245 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:50:52.137624 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:50:52.137624 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:50:52.137624 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:50:52.137624 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:50:52.137624 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:50:52.137624 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:50:52.137624 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:50:52.137624 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:50:52.137624 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:50:52.137624 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:50:52.137624 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:50:52.137624 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 10 11:50:52.137624 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 10 11:50:52.137624 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 10 11:50:52.137624 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 10 11:50:52.138874 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:50:52.138874 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:50:52.138874 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:50:52.138874 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:50:52.138874 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:50:52.138874 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:50:52.138874 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:50:52.138874 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application with Lock( Oct 10 11:50:52.138874 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:50:52.138874 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:50:52.138874 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:50:52.138874 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:50:52.138874 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:50:52.138874 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:50:52.138874 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:50:52.138874 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:50:52.138874 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 10 11:50:52.140104 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 10 11:50:52.140104 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:50:52.140104 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Oct 10 11:50:52.140104 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 605/1237] 10.222.0.46 () {56 vars in 1108 bytes} [Mon Oct 10 11:50:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:50:52.149251 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b7ed20-3f0a-4505-9cb8-ca1ebc6b9b1b None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:50:52.252569 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b7ed20-3f0a-4505-9cb8-ca1ebc6b9b1b None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:50:52.262193 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:50:52.296219 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4736ce80-f4ce-42b4-bb3f-6efe3ee418aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:50:52.296749 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4736ce80-f4ce-42b4-bb3f-6efe3ee418aa None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:50:52.297029 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4736ce80-f4ce-42b4-bb3f-6efe3ee418aa None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:50:52.297303 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4736ce80-f4ce-42b4-bb3f-6efe3ee418aa None admin] PATH_INFO: `/v3/domains` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:50:52.297956 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4736ce80-f4ce-42b4-bb3f-6efe3ee418aa None admin] RBAC: Adding query filter params (name=Default) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:50:52.298261 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4736ce80-f4ce-42b4-bb3f-6efe3ee418aa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:50:52.299671 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4736ce80-f4ce-42b4-bb3f-6efe3ee418aa None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:50:52.313280 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 633/1238] 10.222.0.46 () {56 vars in 1109 bytes} [Mon Oct 10 11:50:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:50:52.322991 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8978af7e-7758-45a5-9374-9eeb167f2245 None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:50:52.448134 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8978af7e-7758-45a5-9374-9eeb167f2245 None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:50:52.486289 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08790455-0df7-415a-92c8-88c2d7081018 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:50:52.487157 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08790455-0df7-415a-92c8-88c2d7081018 None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:50:52.488098 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08790455-0df7-415a-92c8-88c2d7081018 None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:50:52.488385 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08790455-0df7-415a-92c8-88c2d7081018 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:50:52.492026 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:50:52.497867 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08790455-0df7-415a-92c8-88c2d7081018 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:50:52.499632 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08790455-0df7-415a-92c8-88c2d7081018 None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:50:52.512202 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application [None req-08790455-0df7-415a-92c8-88c2d7081018 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 10 11:50:52.512202 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:50:52.512202 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:50:52.512202 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:50:52.512202 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:50:52.512202 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:50:52.512202 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:50:52.512202 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:50:52.512202 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:50:52.512202 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:50:52.512202 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:50:52.512202 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:50:52.512202 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 10 11:50:52.512202 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 10 11:50:52.512202 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 10 11:50:52.512202 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 10 11:50:52.513594 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:50:52.513594 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:50:52.513594 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 10 11:50:52.513594 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 10 11:50:52.513594 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 10 11:50:52.513594 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 10 11:50:52.513594 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:50:52.513594 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:50:52.513594 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:50:52.513594 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:50:52.513594 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:50:52.513594 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application with Lock( Oct 10 11:50:52.513594 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:50:52.513594 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:50:52.513594 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:50:52.513594 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:50:52.513594 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:50:52.513594 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:50:52.514734 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:50:52.514734 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:50:52.514734 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 10 11:50:52.514734 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 10 11:50:52.514734 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 10 11:50:52.514734 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 10 11:50:52.514734 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 10 11:50:52.514734 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 10 11:50:52.514734 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 10 11:50:52.514734 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Oct 10 11:50:52.514734 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 606/1239] 10.222.0.46 () {56 vars in 1108 bytes} [Mon Oct 10 11:50:52 2022] GET /identity/v3/users/placement => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:50:52.520341 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4736ce80-f4ce-42b4-bb3f-6efe3ee418aa None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:50:52.634890 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4736ce80-f4ce-42b4-bb3f-6efe3ee418aa None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:50:52.682605 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9232f7-52e3-413d-9f7b-7a59c63a9979 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:50:52.683437 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9232f7-52e3-413d-9f7b-7a59c63a9979 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:50:52.683818 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9232f7-52e3-413d-9f7b-7a59c63a9979 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:50:52.684189 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9232f7-52e3-413d-9f7b-7a59c63a9979 None admin] PATH_INFO: `/v3/users` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:50:52.685015 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9232f7-52e3-413d-9f7b-7a59c63a9979 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:50:52.685506 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9232f7-52e3-413d-9f7b-7a59c63a9979 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:50:52.687059 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9232f7-52e3-413d-9f7b-7a59c63a9979 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:50:52.709256 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 634/1240] 10.222.0.46 () {56 vars in 1143 bytes} [Mon Oct 10 11:50:52 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:50:52.715528 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08790455-0df7-415a-92c8-88c2d7081018 None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:50:52.821128 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08790455-0df7-415a-92c8-88c2d7081018 None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:50:52.858307 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc807dd-db2d-4129-b0e6-85fc1231c847 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:50:52.858949 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc807dd-db2d-4129-b0e6-85fc1231c847 None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:50:52.859232 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc807dd-db2d-4129-b0e6-85fc1231c847 None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:50:52.859507 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc807dd-db2d-4129-b0e6-85fc1231c847 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:50:52.928326 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:50:52.936027 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc807dd-db2d-4129-b0e6-85fc1231c847 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:50:52.937527 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc807dd-db2d-4129-b0e6-85fc1231c847 None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:50:53.044519 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 607/1241] 10.222.0.46 () {58 vars in 1282 bytes} [Mon Oct 10 11:50:52 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 10 11:50:53.050923 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9232f7-52e3-413d-9f7b-7a59c63a9979 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:50:53.080151 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:50:53.166314 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9232f7-52e3-413d-9f7b-7a59c63a9979 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:50:53.207121 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b72650-fc78-4126-b8f7-cc1c4538ad34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:50:53.207686 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b72650-fc78-4126-b8f7-cc1c4538ad34 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:50:53.207963 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b72650-fc78-4126-b8f7-cc1c4538ad34 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:50:53.208234 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b72650-fc78-4126-b8f7-cc1c4538ad34 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:50:53.219556 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b72650-fc78-4126-b8f7-cc1c4538ad34 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:50:53.221417 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b72650-fc78-4126-b8f7-cc1c4538ad34 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:50:53.237065 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application [None req-86b72650-fc78-4126-b8f7-cc1c4538ad34 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:50:53.237065 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:50:53.237065 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:50:53.237065 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:50:53.237065 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:50:53.237065 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:50:53.237065 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:50:53.237065 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:50:53.237065 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:50:53.237065 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:50:53.237065 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:50:53.237065 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:50:53.237065 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 10 11:50:53.237065 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 10 11:50:53.237065 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 10 11:50:53.237065 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 10 11:50:53.239122 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:50:53.239122 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:50:53.239122 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:50:53.239122 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:50:53.239122 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:50:53.239122 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:50:53.239122 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:50:53.239122 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application with Lock( Oct 10 11:50:53.239122 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:50:53.239122 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:50:53.239122 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:50:53.239122 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:50:53.239122 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:50:53.239122 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:50:53.239122 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:50:53.239122 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:50:53.239122 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 10 11:50:53.241538 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 10 11:50:53.241538 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:50:53.241538 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Oct 10 11:50:53.241538 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 635/1242] 10.222.0.46 () {56 vars in 1108 bytes} [Mon Oct 10 11:50:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 10 11:50:53.246468 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc807dd-db2d-4129-b0e6-85fc1231c847 None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:50:53.349535 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:50:53.360205 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc807dd-db2d-4129-b0e6-85fc1231c847 None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:50:53.411127 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b19729-a6f2-4ddc-a97d-435b3395880b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:50:53.411752 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b19729-a6f2-4ddc-a97d-435b3395880b None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:50:53.412035 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b19729-a6f2-4ddc-a97d-435b3395880b None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:50:53.412313 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b19729-a6f2-4ddc-a97d-435b3395880b None admin] PATH_INFO: `/v3/domains` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:50:53.413004 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b19729-a6f2-4ddc-a97d-435b3395880b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:50:53.413311 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b19729-a6f2-4ddc-a97d-435b3395880b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:50:53.414861 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b19729-a6f2-4ddc-a97d-435b3395880b None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:50:53.427218 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 608/1243] 10.222.0.46 () {56 vars in 1109 bytes} [Mon Oct 10 11:50:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 11:50:53.433175 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b72650-fc78-4126-b8f7-cc1c4538ad34 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:50:53.521219 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:50:53.543582 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b72650-fc78-4126-b8f7-cc1c4538ad34 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:50:53.582112 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806820fd-17da-4744-96ee-60a09f15c5a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:50:53.582818 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806820fd-17da-4744-96ee-60a09f15c5a0 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:50:53.583098 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806820fd-17da-4744-96ee-60a09f15c5a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:50:53.583366 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806820fd-17da-4744-96ee-60a09f15c5a0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:50:53.591219 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806820fd-17da-4744-96ee-60a09f15c5a0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:50:53.592714 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806820fd-17da-4744-96ee-60a09f15c5a0 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:50:53.605347 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application [None req-806820fd-17da-4744-96ee-60a09f15c5a0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 10 11:50:53.605347 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:50:53.605347 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:50:53.605347 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:50:53.605347 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:50:53.605347 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:50:53.605347 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:50:53.605347 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:50:53.605347 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:50:53.605347 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:50:53.605347 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:50:53.605347 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:50:53.605347 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 10 11:50:53.605347 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 10 11:50:53.605347 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 10 11:50:53.606557 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 10 11:50:53.606557 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:50:53.606557 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:50:53.606557 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:50:53.606557 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:50:53.606557 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:50:53.606557 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:50:53.606557 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:50:53.606557 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application with Lock( Oct 10 11:50:53.606557 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:50:53.606557 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:50:53.606557 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:50:53.606557 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:50:53.606557 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:50:53.606557 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:50:53.606557 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:50:53.606557 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:50:53.607689 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 10 11:50:53.607689 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 10 11:50:53.607689 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 10 11:50:53.607689 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 10 11:50:53.607689 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 10 11:50:53.607689 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 10 11:50:53.607689 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 10 11:50:53.607689 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Oct 10 11:50:53.607689 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 636/1244] 10.222.0.46 () {56 vars in 1111 bytes} [Mon Oct 10 11:50:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:50:53.614237 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b19729-a6f2-4ddc-a97d-435b3395880b None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:50:53.765921 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b19729-a6f2-4ddc-a97d-435b3395880b None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:50:53.801230 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b8beeb-60c8-49d7-b599-289a3a1280e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:50:53.801897 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b8beeb-60c8-49d7-b599-289a3a1280e9 None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:50:53.802232 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b8beeb-60c8-49d7-b599-289a3a1280e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:50:53.802551 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b8beeb-60c8-49d7-b599-289a3a1280e9 None admin] PATH_INFO: `/v3/projects` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:50:53.803352 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b8beeb-60c8-49d7-b599-289a3a1280e9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:50:53.803657 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b8beeb-60c8-49d7-b599-289a3a1280e9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:50:53.805115 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b8beeb-60c8-49d7-b599-289a3a1280e9 None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:50:53.818231 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 609/1245] 10.222.0.46 () {56 vars in 1148 bytes} [Mon Oct 10 11:50:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:50:53.824608 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806820fd-17da-4744-96ee-60a09f15c5a0 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:50:53.932791 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806820fd-17da-4744-96ee-60a09f15c5a0 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:50:53.969936 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dec0e89-cce4-47fd-ac18-9daa19205b04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:50:53.970558 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dec0e89-cce4-47fd-ac18-9daa19205b04 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:50:53.970869 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dec0e89-cce4-47fd-ac18-9daa19205b04 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:50:53.971147 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dec0e89-cce4-47fd-ac18-9daa19205b04 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:50:53.972394 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dec0e89-cce4-47fd-ac18-9daa19205b04 None admin] RBAC: Adding query filter params (role.id=3c35be38787740f4834c728a2ce73b6b, scope.project.id=4ad3772f7b6a47c28b7ba37a9316a3c1, user.id=8e8f9bb1776d47169ef7febb57c954cf) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:50:53.972705 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dec0e89-cce4-47fd-ac18-9daa19205b04 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:50:53.974333 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dec0e89-cce4-47fd-ac18-9daa19205b04 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:50:53.980917 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 637/1246] 10.222.0.46 () {56 vars in 1374 bytes} [Mon Oct 10 11:50:53 2022] GET /identity/v3/role_assignments?user.id=8e8f9bb1776d47169ef7febb57c954cf&scope.project.id=4ad3772f7b6a47c28b7ba37a9316a3c1&role.id=3c35be38787740f4834c728a2ce73b6b => generated 608 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:50:56.256964 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4a3a85-49af-4884-aa37-e6b3e1c626b9 None None] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:50:56.257553 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4a3a85-49af-4884-aa37-e6b3e1c626b9 None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:50:56.257553 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4a3a85-49af-4884-aa37-e6b3e1c626b9 None None] PATH_INFO: `/` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:50:56.258872 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 610/1247] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 10 11:50:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:50:56.264172 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbd67f4-584b-4466-ad5d-e2f28c78b940 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:50:56.264585 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbd67f4-584b-4466-ad5d-e2f28c78b940 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:50:56.264738 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbd67f4-584b-4466-ad5d-e2f28c78b940 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:50:56.271849 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:50:56.284631 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-afbd67f4-584b-4466-ad5d-e2f28c78b940 None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:50:56.285806 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-afbd67f4-584b-4466-ad5d-e2f28c78b940 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:50:56.352358 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 638/1248] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 10 11:50:56 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 10 11:50:56.623606 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4a3a85-49af-4884-aa37-e6b3e1c626b9 None None] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:50:56.642439 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:50:56.732904 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4a3a85-49af-4884-aa37-e6b3e1c626b9 None None] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:50:56.770970 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecbc9e3-3b8b-4614-8a23-b6546bf2de72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:50:56.771605 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecbc9e3-3b8b-4614-8a23-b6546bf2de72 None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:50:56.771884 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecbc9e3-3b8b-4614-8a23-b6546bf2de72 None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:50:56.772158 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecbc9e3-3b8b-4614-8a23-b6546bf2de72 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:50:56.782347 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ecbc9e3-3b8b-4614-8a23-b6546bf2de72 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 10 11:50:56.787991 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application [None req-8ecbc9e3-3b8b-4614-8a23-b6546bf2de72 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 10 11:50:56.787991 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:50:56.787991 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:50:56.787991 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:50:56.787991 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:50:56.787991 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:50:56.787991 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:50:56.787991 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:50:56.787991 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:50:56.787991 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:50:56.787991 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:50:56.787991 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:50:56.787991 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 10 11:50:56.787991 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 10 11:50:56.787991 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 10 11:50:56.787991 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 10 11:50:56.789227 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 10 11:50:56.789227 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 10 11:50:56.789227 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 10 11:50:56.789227 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 10 11:50:56.789227 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:50:56.789227 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:50:56.789227 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:50:56.789227 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:50:56.789227 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:50:56.789227 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:50:56.789227 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:50:56.789227 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application with Lock( Oct 10 11:50:56.789227 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:50:56.789227 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:50:56.789227 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:50:56.789227 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:50:56.790410 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:50:56.790410 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:50:56.790410 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:50:56.790410 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:50:56.790410 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 10 11:50:56.790410 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 10 11:50:56.790410 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 10 11:50:56.790410 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 10 11:50:56.790410 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 10 11:50:56.790410 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 10 11:50:56.790410 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 10 11:50:56.790410 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Oct 10 11:50:56.790410 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 611/1249] 10.222.0.46 () {56 vars in 1096 bytes} [Mon Oct 10 11:50:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:50:56.795355 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbd67f4-584b-4466-ad5d-e2f28c78b940 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:50:56.894164 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbd67f4-584b-4466-ad5d-e2f28c78b940 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:50:56.923026 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:50:56.928024 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab37673-8db0-4329-9b89-2076fb3e5b79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:50:56.928667 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab37673-8db0-4329-9b89-2076fb3e5b79 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:50:56.929031 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab37673-8db0-4329-9b89-2076fb3e5b79 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:50:56.929346 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab37673-8db0-4329-9b89-2076fb3e5b79 None admin] PATH_INFO: `/v3/roles` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:50:56.929974 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab37673-8db0-4329-9b89-2076fb3e5b79 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:50:56.930338 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab37673-8db0-4329-9b89-2076fb3e5b79 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:50:56.931693 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab37673-8db0-4329-9b89-2076fb3e5b79 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:50:56.939919 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 639/1250] 10.222.0.46 () {56 vars in 1099 bytes} [Mon Oct 10 11:50:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:50:56.947426 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecbc9e3-3b8b-4614-8a23-b6546bf2de72 None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:50:57.028232 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:50:57.051193 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecbc9e3-3b8b-4614-8a23-b6546bf2de72 None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:50:57.086483 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c773e49a-c6f9-4e96-bcef-ed028d2f47dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:50:57.087221 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c773e49a-c6f9-4e96-bcef-ed028d2f47dd None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:50:57.087502 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c773e49a-c6f9-4e96-bcef-ed028d2f47dd None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:50:57.087772 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c773e49a-c6f9-4e96-bcef-ed028d2f47dd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:50:57.098390 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c773e49a-c6f9-4e96-bcef-ed028d2f47dd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:50:57.099954 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c773e49a-c6f9-4e96-bcef-ed028d2f47dd None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:50:57.115909 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application [None req-c773e49a-c6f9-4e96-bcef-ed028d2f47dd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:50:57.115909 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:50:57.115909 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:50:57.115909 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:50:57.115909 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:50:57.115909 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:50:57.115909 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:50:57.115909 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:50:57.115909 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:50:57.115909 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:50:57.115909 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:50:57.115909 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:50:57.115909 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 10 11:50:57.115909 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 10 11:50:57.115909 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 10 11:50:57.115909 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 10 11:50:57.125948 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:50:57.125948 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:50:57.125948 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:50:57.125948 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:50:57.125948 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:50:57.125948 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:50:57.125948 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:50:57.125948 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application with Lock( Oct 10 11:50:57.125948 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:50:57.125948 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:50:57.125948 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:50:57.125948 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:50:57.125948 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:50:57.125948 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:50:57.125948 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:50:57.125948 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:50:57.125948 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 10 11:50:57.127058 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 10 11:50:57.127058 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:50:57.127058 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Oct 10 11:50:57.127058 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 612/1251] 10.222.0.46 () {56 vars in 1108 bytes} [Mon Oct 10 11:50:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:50:57.127346 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab37673-8db0-4329-9b89-2076fb3e5b79 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:50:57.229500 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab37673-8db0-4329-9b89-2076fb3e5b79 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:50:57.265348 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48e650c-d3bc-4fa5-80a4-6389a5aea7a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:50:57.265992 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48e650c-d3bc-4fa5-80a4-6389a5aea7a8 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:50:57.266230 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48e650c-d3bc-4fa5-80a4-6389a5aea7a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:50:57.266499 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48e650c-d3bc-4fa5-80a4-6389a5aea7a8 None admin] PATH_INFO: `/v3/domains` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:50:57.267181 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48e650c-d3bc-4fa5-80a4-6389a5aea7a8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:50:57.267500 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48e650c-d3bc-4fa5-80a4-6389a5aea7a8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:50:57.269497 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48e650c-d3bc-4fa5-80a4-6389a5aea7a8 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:50:57.281102 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 640/1252] 10.222.0.46 () {56 vars in 1109 bytes} [Mon Oct 10 11:50:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:50:57.287079 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c773e49a-c6f9-4e96-bcef-ed028d2f47dd None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:50:57.389374 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c773e49a-c6f9-4e96-bcef-ed028d2f47dd None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:50:57.415710 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:50:57.423495 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c783f92-b494-48dc-8f7f-b4d672285506 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:50:57.424128 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c783f92-b494-48dc-8f7f-b4d672285506 None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:50:57.424411 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c783f92-b494-48dc-8f7f-b4d672285506 None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:50:57.424688 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c783f92-b494-48dc-8f7f-b4d672285506 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:50:57.432888 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c783f92-b494-48dc-8f7f-b4d672285506 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:50:57.434330 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c783f92-b494-48dc-8f7f-b4d672285506 None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:50:57.446335 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application [None req-4c783f92-b494-48dc-8f7f-b4d672285506 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 10 11:50:57.446335 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:50:57.446335 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:50:57.446335 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:50:57.446335 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:50:57.446335 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:50:57.446335 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:50:57.446335 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:50:57.446335 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:50:57.446335 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:50:57.446335 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:50:57.446335 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:50:57.446335 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 10 11:50:57.446335 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 10 11:50:57.446335 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 10 11:50:57.446335 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 10 11:50:57.447537 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:50:57.447537 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:50:57.447537 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 10 11:50:57.447537 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 10 11:50:57.447537 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 10 11:50:57.447537 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 10 11:50:57.447537 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:50:57.447537 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:50:57.447537 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:50:57.447537 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:50:57.447537 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:50:57.447537 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application with Lock( Oct 10 11:50:57.447537 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:50:57.447537 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:50:57.447537 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:50:57.447537 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:50:57.447537 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:50:57.447537 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:50:57.448729 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:50:57.448729 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:50:57.448729 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 10 11:50:57.448729 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 10 11:50:57.448729 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 10 11:50:57.448729 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 10 11:50:57.448729 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 10 11:50:57.448729 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 10 11:50:57.448729 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 10 11:50:57.448729 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Oct 10 11:50:57.448729 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 613/1253] 10.222.0.46 () {56 vars in 1108 bytes} [Mon Oct 10 11:50:57 2022] GET /identity/v3/users/placement => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 10 11:50:57.453587 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48e650c-d3bc-4fa5-80a4-6389a5aea7a8 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:50:57.521246 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:50:57.566923 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48e650c-d3bc-4fa5-80a4-6389a5aea7a8 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:50:57.601582 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc03717-327b-4c84-9313-64376bbab0e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:50:57.602440 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc03717-327b-4c84-9313-64376bbab0e5 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:50:57.603290 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc03717-327b-4c84-9313-64376bbab0e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:50:57.603564 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc03717-327b-4c84-9313-64376bbab0e5 None admin] PATH_INFO: `/v3/users` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:50:57.604326 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc03717-327b-4c84-9313-64376bbab0e5 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:50:57.604637 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc03717-327b-4c84-9313-64376bbab0e5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:50:57.606119 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc03717-327b-4c84-9313-64376bbab0e5 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:50:57.625319 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 641/1254] 10.222.0.46 () {56 vars in 1143 bytes} [Mon Oct 10 11:50:57 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:50:57.631792 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c783f92-b494-48dc-8f7f-b4d672285506 None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:50:57.741046 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c783f92-b494-48dc-8f7f-b4d672285506 None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:50:57.777183 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d3a675-7bc4-4afc-be12-44a3a8772bd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:50:57.777945 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d3a675-7bc4-4afc-be12-44a3a8772bd9 None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:50:57.778204 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d3a675-7bc4-4afc-be12-44a3a8772bd9 None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:50:57.778480 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d3a675-7bc4-4afc-be12-44a3a8772bd9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:50:57.845718 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:50:57.858821 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d3a675-7bc4-4afc-be12-44a3a8772bd9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:50:57.860288 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d3a675-7bc4-4afc-be12-44a3a8772bd9 None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:50:57.960491 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 614/1255] 10.222.0.46 () {58 vars in 1282 bytes} [Mon Oct 10 11:50:57 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 10 11:50:57.966594 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc03717-327b-4c84-9313-64376bbab0e5 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:50:58.066965 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc03717-327b-4c84-9313-64376bbab0e5 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:50:58.082599 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:50:58.104168 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e70c97-64bf-43dc-b042-74cf38e399b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:50:58.105041 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e70c97-64bf-43dc-b042-74cf38e399b9 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:50:58.105173 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e70c97-64bf-43dc-b042-74cf38e399b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:50:58.105344 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e70c97-64bf-43dc-b042-74cf38e399b9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:50:58.115930 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e70c97-64bf-43dc-b042-74cf38e399b9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:50:58.117456 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e70c97-64bf-43dc-b042-74cf38e399b9 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:50:58.132732 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application [None req-d8e70c97-64bf-43dc-b042-74cf38e399b9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:50:58.132732 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:50:58.132732 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:50:58.132732 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:50:58.132732 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:50:58.132732 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:50:58.132732 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:50:58.132732 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:50:58.132732 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:50:58.132732 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:50:58.132732 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:50:58.132732 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:50:58.132732 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 10 11:50:58.132732 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 10 11:50:58.132732 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 10 11:50:58.132732 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 10 11:50:58.134186 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:50:58.134186 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:50:58.134186 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:50:58.134186 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:50:58.134186 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:50:58.134186 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:50:58.134186 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:50:58.134186 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application with Lock( Oct 10 11:50:58.134186 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:50:58.134186 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:50:58.134186 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:50:58.134186 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:50:58.134186 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:50:58.134186 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:50:58.134186 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:50:58.134186 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:50:58.134186 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 10 11:50:58.135376 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 10 11:50:58.135376 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:50:58.135376 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Oct 10 11:50:58.135376 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 642/1256] 10.222.0.46 () {56 vars in 1108 bytes} [Mon Oct 10 11:50:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:50:58.139638 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d3a675-7bc4-4afc-be12-44a3a8772bd9 None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:50:58.228348 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:50:58.241087 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d3a675-7bc4-4afc-be12-44a3a8772bd9 None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:50:58.277044 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802135c9-0648-4342-b365-896127ef2ad1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:50:58.277703 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802135c9-0648-4342-b365-896127ef2ad1 None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:50:58.277969 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802135c9-0648-4342-b365-896127ef2ad1 None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:50:58.278244 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802135c9-0648-4342-b365-896127ef2ad1 None admin] PATH_INFO: `/v3/domains` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:50:58.278925 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802135c9-0648-4342-b365-896127ef2ad1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:50:58.279234 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802135c9-0648-4342-b365-896127ef2ad1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:50:58.280663 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802135c9-0648-4342-b365-896127ef2ad1 None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:50:58.292717 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 615/1257] 10.222.0.46 () {56 vars in 1109 bytes} [Mon Oct 10 11:50:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:50:58.298887 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e70c97-64bf-43dc-b042-74cf38e399b9 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:50:58.399183 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e70c97-64bf-43dc-b042-74cf38e399b9 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:50:58.433454 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8bcff3-e11e-4564-8a67-79410ecf17d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:50:58.434083 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8bcff3-e11e-4564-8a67-79410ecf17d1 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:50:58.434366 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8bcff3-e11e-4564-8a67-79410ecf17d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:50:58.434641 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8bcff3-e11e-4564-8a67-79410ecf17d1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:50:58.442161 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8bcff3-e11e-4564-8a67-79410ecf17d1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:50:58.443628 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8bcff3-e11e-4564-8a67-79410ecf17d1 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:50:58.455502 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application [None req-ac8bcff3-e11e-4564-8a67-79410ecf17d1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 10 11:50:58.455502 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:50:58.455502 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:50:58.455502 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:50:58.455502 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:50:58.455502 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:50:58.455502 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:50:58.455502 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:50:58.455502 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:50:58.455502 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:50:58.455502 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:50:58.455502 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:50:58.455502 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 10 11:50:58.455502 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 10 11:50:58.455502 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 10 11:50:58.464497 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 10 11:50:58.464497 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:50:58.464497 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:50:58.464497 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:50:58.464497 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:50:58.464497 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:50:58.464497 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:50:58.464497 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:50:58.464497 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application with Lock( Oct 10 11:50:58.464497 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:50:58.464497 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:50:58.464497 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:50:58.464497 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:50:58.464497 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:50:58.464497 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:50:58.464497 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:50:58.464497 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:50:58.465605 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 10 11:50:58.465605 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 10 11:50:58.465605 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 10 11:50:58.465605 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 10 11:50:58.465605 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 10 11:50:58.465605 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 10 11:50:58.465605 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 10 11:50:58.465605 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Oct 10 11:50:58.465605 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 643/1258] 10.222.0.46 () {56 vars in 1111 bytes} [Mon Oct 10 11:50:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:50:58.466215 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802135c9-0648-4342-b365-896127ef2ad1 None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:50:58.570886 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802135c9-0648-4342-b365-896127ef2ad1 None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:50:58.605404 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ddb6570-0437-48ad-a5bb-f5a4617d63cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:50:58.606025 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddb6570-0437-48ad-a5bb-f5a4617d63cc None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:50:58.606304 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddb6570-0437-48ad-a5bb-f5a4617d63cc None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:50:58.606578 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddb6570-0437-48ad-a5bb-f5a4617d63cc None admin] PATH_INFO: `/v3/projects` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:50:58.607315 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ddb6570-0437-48ad-a5bb-f5a4617d63cc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:50:58.607622 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ddb6570-0437-48ad-a5bb-f5a4617d63cc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:50:58.609202 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ddb6570-0437-48ad-a5bb-f5a4617d63cc None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:50:58.623204 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 616/1259] 10.222.0.46 () {56 vars in 1148 bytes} [Mon Oct 10 11:50:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:50:58.630073 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8bcff3-e11e-4564-8a67-79410ecf17d1 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:50:58.656218 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:50:58.733580 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8bcff3-e11e-4564-8a67-79410ecf17d1 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:50:58.771630 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f5e760-977c-4407-af06-89cc7f32bd86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:50:58.772266 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f5e760-977c-4407-af06-89cc7f32bd86 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:50:58.772538 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f5e760-977c-4407-af06-89cc7f32bd86 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:50:58.772809 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f5e760-977c-4407-af06-89cc7f32bd86 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:50:58.773548 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f5e760-977c-4407-af06-89cc7f32bd86 None admin] RBAC: Adding query filter params (role.id=0d822a0674b6495fac623877a3b38025, scope.project.id=4ad3772f7b6a47c28b7ba37a9316a3c1, user.id=8e8f9bb1776d47169ef7febb57c954cf) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:50:58.773854 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f5e760-977c-4407-af06-89cc7f32bd86 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:50:58.775341 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f5e760-977c-4407-af06-89cc7f32bd86 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:50:58.781499 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 644/1260] 10.222.0.46 () {56 vars in 1374 bytes} [Mon Oct 10 11:50:58 2022] GET /identity/v3/role_assignments?user.id=8e8f9bb1776d47169ef7febb57c954cf&scope.project.id=4ad3772f7b6a47c28b7ba37a9316a3c1&role.id=0d822a0674b6495fac623877a3b38025 => generated 259 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:51:00.960583 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efbec45-ac9c-43f7-9959-026be5452b74 None None] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:51:00.961116 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efbec45-ac9c-43f7-9959-026be5452b74 None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:51:00.961116 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efbec45-ac9c-43f7-9959-026be5452b74 None None] PATH_INFO: `/` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:51:00.962477 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 617/1261] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 10 11:51:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:51:00.968057 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b5706a-0755-46fd-9fed-1a4d99f0df1e None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:51:00.968513 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b5706a-0755-46fd-9fed-1a4d99f0df1e None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:51:00.968614 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b5706a-0755-46fd-9fed-1a4d99f0df1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:51:00.987214 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-51b5706a-0755-46fd-9fed-1a4d99f0df1e None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:51:00.990303 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-51b5706a-0755-46fd-9fed-1a4d99f0df1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:51:01.055436 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 645/1262] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 10 11:51:00 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 10 11:51:01.324198 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efbec45-ac9c-43f7-9959-026be5452b74 None None] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:51:01.336066 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:51:01.429676 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efbec45-ac9c-43f7-9959-026be5452b74 None None] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:51:01.465634 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b3c569-0917-4e02-b403-38c34627ab40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:51:01.466350 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b3c569-0917-4e02-b403-38c34627ab40 None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:51:01.466632 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b3c569-0917-4e02-b403-38c34627ab40 None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:51:01.466935 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b3c569-0917-4e02-b403-38c34627ab40 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:51:01.477535 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b3c569-0917-4e02-b403-38c34627ab40 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 10 11:51:01.482638 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application [None req-c4b3c569-0917-4e02-b403-38c34627ab40 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 10 11:51:01.482638 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:51:01.482638 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:51:01.482638 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:51:01.482638 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:51:01.482638 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:51:01.482638 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:51:01.482638 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:51:01.482638 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:51:01.482638 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:51:01.482638 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:51:01.482638 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:51:01.482638 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 10 11:51:01.482638 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 10 11:51:01.482638 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 10 11:51:01.482638 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 10 11:51:01.484324 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 10 11:51:01.484324 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 10 11:51:01.484324 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 10 11:51:01.484324 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 10 11:51:01.484324 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:51:01.484324 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:51:01.484324 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:51:01.484324 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:51:01.484324 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:51:01.484324 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:51:01.484324 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:51:01.484324 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application with Lock( Oct 10 11:51:01.484324 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:51:01.484324 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:51:01.484324 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:51:01.484324 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:51:01.485165 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:51:01.485165 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:51:01.485165 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:51:01.485165 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:51:01.485165 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 10 11:51:01.485165 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 10 11:51:01.485165 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 10 11:51:01.485165 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 10 11:51:01.485165 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 10 11:51:01.485165 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 10 11:51:01.485165 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 10 11:51:01.485165 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Oct 10 11:51:01.485165 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 618/1263] 10.222.0.46 () {56 vars in 1096 bytes} [Mon Oct 10 11:51:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 10 11:51:01.489993 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b5706a-0755-46fd-9fed-1a4d99f0df1e None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:51:01.508701 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:51:01.587060 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b5706a-0755-46fd-9fed-1a4d99f0df1e None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:51:01.620038 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b96f605-7f92-437c-b6ad-b84af3b5deaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:51:01.620639 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b96f605-7f92-437c-b6ad-b84af3b5deaa None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:51:01.620913 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b96f605-7f92-437c-b6ad-b84af3b5deaa None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:51:01.621184 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b96f605-7f92-437c-b6ad-b84af3b5deaa None admin] PATH_INFO: `/v3/roles` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:51:01.621836 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b96f605-7f92-437c-b6ad-b84af3b5deaa None admin] RBAC: Adding query filter params (name=admin) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:51:01.622131 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b96f605-7f92-437c-b6ad-b84af3b5deaa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:51:01.623505 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b96f605-7f92-437c-b6ad-b84af3b5deaa None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:51:01.632870 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 646/1264] 10.222.0.46 () {56 vars in 1099 bytes} [Mon Oct 10 11:51:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:51:01.638877 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b3c569-0917-4e02-b403-38c34627ab40 None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:51:01.707870 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:51:01.741013 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b3c569-0917-4e02-b403-38c34627ab40 None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:51:01.775251 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf9851e-c2a2-4871-b8db-a9f7666de3e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:51:01.778973 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf9851e-c2a2-4871-b8db-a9f7666de3e6 None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:51:01.778973 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf9851e-c2a2-4871-b8db-a9f7666de3e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:51:01.778973 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf9851e-c2a2-4871-b8db-a9f7666de3e6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:51:01.787556 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf9851e-c2a2-4871-b8db-a9f7666de3e6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:51:01.789037 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf9851e-c2a2-4871-b8db-a9f7666de3e6 None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:51:01.803786 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application [None req-1bf9851e-c2a2-4871-b8db-a9f7666de3e6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:51:01.803786 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:51:01.803786 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:51:01.803786 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:51:01.803786 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:51:01.803786 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:51:01.803786 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:51:01.803786 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:51:01.803786 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:51:01.803786 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:51:01.803786 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:51:01.803786 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:51:01.803786 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 10 11:51:01.803786 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 10 11:51:01.803786 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 10 11:51:01.803786 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 10 11:51:01.805822 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:51:01.805822 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:51:01.805822 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:51:01.805822 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:51:01.805822 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:51:01.805822 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:51:01.805822 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:51:01.805822 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application with Lock( Oct 10 11:51:01.805822 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:51:01.805822 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:51:01.805822 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:51:01.805822 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:51:01.805822 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:51:01.805822 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:51:01.805822 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:51:01.805822 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:51:01.805822 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 10 11:51:01.807040 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 10 11:51:01.807040 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:51:01.807040 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Oct 10 11:51:01.807040 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 619/1265] 10.222.0.46 () {56 vars in 1108 bytes} [Mon Oct 10 11:51:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:51:01.811077 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b96f605-7f92-437c-b6ad-b84af3b5deaa None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:51:01.894141 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:51:01.912847 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b96f605-7f92-437c-b6ad-b84af3b5deaa None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:51:01.948847 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a53e3c-c8e8-4764-980c-04cf349bb846 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:51:01.949522 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a53e3c-c8e8-4764-980c-04cf349bb846 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:51:01.949775 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a53e3c-c8e8-4764-980c-04cf349bb846 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:51:01.950027 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a53e3c-c8e8-4764-980c-04cf349bb846 None admin] PATH_INFO: `/v3/domains` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:51:01.950689 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a53e3c-c8e8-4764-980c-04cf349bb846 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:51:01.950997 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a53e3c-c8e8-4764-980c-04cf349bb846 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:51:01.952350 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a53e3c-c8e8-4764-980c-04cf349bb846 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:51:01.963422 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 647/1266] 10.222.0.46 () {56 vars in 1109 bytes} [Mon Oct 10 11:51:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:51:01.969418 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf9851e-c2a2-4871-b8db-a9f7666de3e6 None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:51:02.079299 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf9851e-c2a2-4871-b8db-a9f7666de3e6 None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:51:02.099264 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:51:02.113438 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a911c33-1002-40cb-bfe4-68400f0792c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:51:02.114328 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a911c33-1002-40cb-bfe4-68400f0792c2 None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:51:02.114606 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a911c33-1002-40cb-bfe4-68400f0792c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:51:02.114936 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a911c33-1002-40cb-bfe4-68400f0792c2 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:51:02.122403 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a911c33-1002-40cb-bfe4-68400f0792c2 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:51:02.123924 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a911c33-1002-40cb-bfe4-68400f0792c2 None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:51:02.135623 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application [None req-2a911c33-1002-40cb-bfe4-68400f0792c2 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 10 11:51:02.135623 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:51:02.135623 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:51:02.135623 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:51:02.135623 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:51:02.135623 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:51:02.135623 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:51:02.135623 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:51:02.135623 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:51:02.135623 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:51:02.135623 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:51:02.135623 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:51:02.135623 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 10 11:51:02.135623 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 10 11:51:02.135623 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 10 11:51:02.135623 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 10 11:51:02.136974 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:51:02.136974 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:51:02.136974 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 10 11:51:02.136974 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 10 11:51:02.136974 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 10 11:51:02.136974 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 10 11:51:02.136974 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:51:02.136974 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:51:02.136974 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:51:02.136974 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:51:02.136974 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:51:02.136974 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application with Lock( Oct 10 11:51:02.136974 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:51:02.136974 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:51:02.136974 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:51:02.136974 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:51:02.136974 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:51:02.136974 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:51:02.138326 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:51:02.138326 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:51:02.138326 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 10 11:51:02.138326 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 10 11:51:02.138326 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 10 11:51:02.138326 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 10 11:51:02.138326 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 10 11:51:02.138326 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 10 11:51:02.138326 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 10 11:51:02.138326 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Oct 10 11:51:02.138326 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 620/1267] 10.222.0.46 () {56 vars in 1108 bytes} [Mon Oct 10 11:51:01 2022] GET /identity/v3/users/placement => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:51:02.142732 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a53e3c-c8e8-4764-980c-04cf349bb846 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:51:02.244930 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a53e3c-c8e8-4764-980c-04cf349bb846 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:51:02.277866 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1190f886-8918-4118-8285-e53805721e41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:51:02.278470 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1190f886-8918-4118-8285-e53805721e41 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:51:02.278783 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1190f886-8918-4118-8285-e53805721e41 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:51:02.279080 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1190f886-8918-4118-8285-e53805721e41 None admin] PATH_INFO: `/v3/users` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:51:02.279793 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1190f886-8918-4118-8285-e53805721e41 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:51:02.280102 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1190f886-8918-4118-8285-e53805721e41 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:51:02.281523 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1190f886-8918-4118-8285-e53805721e41 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:51:02.290987 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:51:02.298829 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 648/1268] 10.222.0.46 () {56 vars in 1143 bytes} [Mon Oct 10 11:51:02 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:51:02.305366 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a911c33-1002-40cb-bfe4-68400f0792c2 None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:51:02.420845 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a911c33-1002-40cb-bfe4-68400f0792c2 None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:51:02.457883 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01129722-a601-4746-beea-23003e670dd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:51:02.458506 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01129722-a601-4746-beea-23003e670dd0 None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:51:02.458816 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01129722-a601-4746-beea-23003e670dd0 None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:51:02.459093 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01129722-a601-4746-beea-23003e670dd0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:51:02.469541 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01129722-a601-4746-beea-23003e670dd0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:51:02.472348 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01129722-a601-4746-beea-23003e670dd0 None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:51:02.485861 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application [None req-01129722-a601-4746-beea-23003e670dd0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:51:02.485861 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:51:02.485861 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:51:02.485861 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:51:02.485861 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:51:02.485861 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:51:02.485861 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:51:02.485861 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:51:02.485861 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:51:02.485861 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:51:02.485861 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:51:02.485861 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:51:02.485861 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 10 11:51:02.485861 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 10 11:51:02.485861 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 10 11:51:02.485861 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 10 11:51:02.487068 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:51:02.487068 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:51:02.487068 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:51:02.487068 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:51:02.487068 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:51:02.487068 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:51:02.487068 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:51:02.487068 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application with Lock( Oct 10 11:51:02.487068 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:51:02.487068 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:51:02.487068 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:51:02.487068 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:51:02.487068 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:51:02.487068 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:51:02.487068 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:51:02.487068 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:51:02.487068 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 10 11:51:02.488770 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 10 11:51:02.488770 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:51:02.488770 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Oct 10 11:51:02.488770 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 621/1269] 10.222.0.46 () {56 vars in 1108 bytes} [Mon Oct 10 11:51:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:51:02.494586 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1190f886-8918-4118-8285-e53805721e41 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:51:02.592656 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1190f886-8918-4118-8285-e53805721e41 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:51:02.626494 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a068bcf-eeca-4694-ab99-39e4dd8a8c0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:51:02.627128 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a068bcf-eeca-4694-ab99-39e4dd8a8c0e None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:51:02.627408 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a068bcf-eeca-4694-ab99-39e4dd8a8c0e None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:51:02.627685 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a068bcf-eeca-4694-ab99-39e4dd8a8c0e None admin] PATH_INFO: `/v3/domains` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:51:02.628316 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a068bcf-eeca-4694-ab99-39e4dd8a8c0e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:51:02.628647 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a068bcf-eeca-4694-ab99-39e4dd8a8c0e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:51:02.630004 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a068bcf-eeca-4694-ab99-39e4dd8a8c0e None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:51:02.641385 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 649/1270] 10.222.0.46 () {56 vars in 1109 bytes} [Mon Oct 10 11:51:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:51:02.647702 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01129722-a601-4746-beea-23003e670dd0 None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:51:02.680077 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:51:02.747543 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01129722-a601-4746-beea-23003e670dd0 None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:51:02.781425 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114e7428-35e4-4cde-9c19-32dfc6802736 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:51:02.782047 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114e7428-35e4-4cde-9c19-32dfc6802736 None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:51:02.782329 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114e7428-35e4-4cde-9c19-32dfc6802736 None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:51:02.782602 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114e7428-35e4-4cde-9c19-32dfc6802736 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:51:02.789983 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114e7428-35e4-4cde-9c19-32dfc6802736 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:51:02.791747 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114e7428-35e4-4cde-9c19-32dfc6802736 None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:51:02.804043 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application [None req-114e7428-35e4-4cde-9c19-32dfc6802736 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 10 11:51:02.804043 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:51:02.804043 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:51:02.804043 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:51:02.804043 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:51:02.804043 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:51:02.804043 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:51:02.804043 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:51:02.804043 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:51:02.804043 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:51:02.804043 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:51:02.804043 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:51:02.804043 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 10 11:51:02.804043 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 10 11:51:02.804043 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 10 11:51:02.805145 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 10 11:51:02.805145 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:51:02.805145 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:51:02.805145 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:51:02.805145 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:51:02.805145 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:51:02.805145 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:51:02.805145 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:51:02.805145 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application with Lock( Oct 10 11:51:02.805145 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:51:02.805145 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:51:02.805145 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:51:02.805145 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:51:02.805145 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:51:02.805145 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:51:02.805145 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:51:02.805145 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:51:02.806255 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 10 11:51:02.806255 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 10 11:51:02.806255 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 10 11:51:02.806255 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 10 11:51:02.806255 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 10 11:51:02.806255 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 10 11:51:02.806255 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 10 11:51:02.806255 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Oct 10 11:51:02.806255 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 622/1271] 10.222.0.46 () {56 vars in 1111 bytes} [Mon Oct 10 11:51:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:51:02.811113 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a068bcf-eeca-4694-ab99-39e4dd8a8c0e None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:51:02.867619 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:51:02.906982 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a068bcf-eeca-4694-ab99-39e4dd8a8c0e None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:51:02.938955 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5751a09-1c01-48e0-86e5-492efaf431d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:51:02.939568 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5751a09-1c01-48e0-86e5-492efaf431d8 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:51:02.939854 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5751a09-1c01-48e0-86e5-492efaf431d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:51:02.940182 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5751a09-1c01-48e0-86e5-492efaf431d8 None admin] PATH_INFO: `/v3/projects` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:51:02.940859 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5751a09-1c01-48e0-86e5-492efaf431d8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:51:02.941653 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5751a09-1c01-48e0-86e5-492efaf431d8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:51:02.943168 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5751a09-1c01-48e0-86e5-492efaf431d8 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:51:02.955335 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 650/1272] 10.222.0.46 () {56 vars in 1148 bytes} [Mon Oct 10 11:51:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:51:02.962104 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114e7428-35e4-4cde-9c19-32dfc6802736 None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:51:03.052249 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:51:03.060888 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114e7428-35e4-4cde-9c19-32dfc6802736 None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:51:03.095661 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89f18e5-14cd-421d-a875-adb603e86500 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:51:03.096326 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89f18e5-14cd-421d-a875-adb603e86500 None admin] REQUEST_METHOD: `PUT` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:51:03.096610 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89f18e5-14cd-421d-a875-adb603e86500 None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:51:03.096887 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89f18e5-14cd-421d-a875-adb603e86500 None admin] PATH_INFO: `/v3/projects/4ad3772f7b6a47c28b7ba37a9316a3c1/users/8e8f9bb1776d47169ef7febb57c954cf/roles/0d822a0674b6495fac623877a3b38025` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:51:03.098843 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89f18e5-14cd-421d-a875-adb603e86500 None admin] RBAC: Authorizing `identity:create_grant(project_id=4ad3772f7b6a47c28b7ba37a9316a3c1, user_id=8e8f9bb1776d47169ef7febb57c954cf, role_id=0d822a0674b6495fac623877a3b38025)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:51:03.100459 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89f18e5-14cd-421d-a875-adb603e86500 None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:51:03.181871 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 623/1273] 10.222.0.46 () {58 vars in 1439 bytes} [Mon Oct 10 11:51:02 2022] PUT /identity/v3/projects/4ad3772f7b6a47c28b7ba37a9316a3c1/users/8e8f9bb1776d47169ef7febb57c954cf/roles/0d822a0674b6495fac623877a3b38025 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 11:51:05.361566 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0030db84-5e8b-4fed-b7de-de3853adff9f None None] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:51:05.362180 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0030db84-5e8b-4fed-b7de-de3853adff9f None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:51:05.362180 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0030db84-5e8b-4fed-b7de-de3853adff9f None None] PATH_INFO: `/` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:51:05.364393 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 651/1274] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 10 11:51:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:51:05.369618 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117a5de5-7dd5-4b43-a9ae-1d566257058d None None] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:51:05.370038 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117a5de5-7dd5-4b43-a9ae-1d566257058d None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:51:05.370174 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117a5de5-7dd5-4b43-a9ae-1d566257058d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:51:05.388424 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.auth.core [None req-117a5de5-7dd5-4b43-a9ae-1d566257058d None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:51:05.389656 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-117a5de5-7dd5-4b43-a9ae-1d566257058d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:51:05.461260 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 624/1275] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 10 11:51:05 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 10 11:51:05.734759 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0030db84-5e8b-4fed-b7de-de3853adff9f None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:51:05.836677 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0030db84-5e8b-4fed-b7de-de3853adff9f None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:51:05.854440 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:51:05.872514 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e74993a-9100-4a82-a8af-30ebb9c455ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:51:05.873144 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e74993a-9100-4a82-a8af-30ebb9c455ee None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:51:05.873423 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e74993a-9100-4a82-a8af-30ebb9c455ee None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:51:05.873694 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e74993a-9100-4a82-a8af-30ebb9c455ee None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:51:05.883650 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e74993a-9100-4a82-a8af-30ebb9c455ee None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 10 11:51:05.888446 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application [None req-4e74993a-9100-4a82-a8af-30ebb9c455ee None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 10 11:51:05.888446 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:51:05.888446 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:51:05.888446 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:51:05.888446 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:51:05.888446 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:51:05.888446 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:51:05.888446 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:51:05.888446 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:51:05.888446 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:51:05.888446 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:51:05.888446 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:51:05.888446 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 10 11:51:05.888446 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 10 11:51:05.888446 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 10 11:51:05.888446 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 10 11:51:05.890510 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 10 11:51:05.890510 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 10 11:51:05.890510 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 10 11:51:05.890510 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 10 11:51:05.890510 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:51:05.890510 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:51:05.890510 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:51:05.890510 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:51:05.890510 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:51:05.890510 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:51:05.890510 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:51:05.890510 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application with Lock( Oct 10 11:51:05.890510 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:51:05.890510 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:51:05.890510 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:51:05.890510 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:51:05.891402 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:51:05.891402 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:51:05.891402 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:51:05.891402 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:51:05.891402 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 10 11:51:05.891402 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 10 11:51:05.891402 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 10 11:51:05.891402 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 10 11:51:05.891402 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 10 11:51:05.891402 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 10 11:51:05.891402 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 10 11:51:05.891402 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Oct 10 11:51:05.891402 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 652/1276] 10.222.0.46 () {56 vars in 1096 bytes} [Mon Oct 10 11:51:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:51:05.896145 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117a5de5-7dd5-4b43-a9ae-1d566257058d None None] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:51:05.974036 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:51:05.991161 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117a5de5-7dd5-4b43-a9ae-1d566257058d None None] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:51:06.032699 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95fc617-0b1a-41a2-806e-34261473fd8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:51:06.033369 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95fc617-0b1a-41a2-806e-34261473fd8d None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:51:06.033726 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95fc617-0b1a-41a2-806e-34261473fd8d None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:51:06.034010 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95fc617-0b1a-41a2-806e-34261473fd8d None admin] PATH_INFO: `/v3/roles` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:51:06.034708 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95fc617-0b1a-41a2-806e-34261473fd8d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:51:06.035016 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95fc617-0b1a-41a2-806e-34261473fd8d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:51:06.036450 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95fc617-0b1a-41a2-806e-34261473fd8d None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:51:06.044497 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 625/1277] 10.222.0.46 () {56 vars in 1099 bytes} [Mon Oct 10 11:51:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:51:06.050573 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e74993a-9100-4a82-a8af-30ebb9c455ee None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:51:06.150618 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e74993a-9100-4a82-a8af-30ebb9c455ee None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:51:06.184908 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe5f0bc-34f6-4c01-b47a-33294afb6cdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:51:06.193891 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe5f0bc-34f6-4c01-b47a-33294afb6cdb None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:51:06.193891 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe5f0bc-34f6-4c01-b47a-33294afb6cdb None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:51:06.193891 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe5f0bc-34f6-4c01-b47a-33294afb6cdb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:51:06.196654 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe5f0bc-34f6-4c01-b47a-33294afb6cdb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:51:06.198207 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe5f0bc-34f6-4c01-b47a-33294afb6cdb None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:51:06.213470 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application [None req-9fe5f0bc-34f6-4c01-b47a-33294afb6cdb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:51:06.213470 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:51:06.213470 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:51:06.213470 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:51:06.213470 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:51:06.213470 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:51:06.213470 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:51:06.213470 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:51:06.213470 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:51:06.213470 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:51:06.213470 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:51:06.213470 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:51:06.213470 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 10 11:51:06.213470 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 10 11:51:06.213470 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 10 11:51:06.213470 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 10 11:51:06.214603 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:51:06.214603 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:51:06.214603 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:51:06.214603 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:51:06.214603 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:51:06.214603 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:51:06.214603 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:51:06.214603 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application with Lock( Oct 10 11:51:06.214603 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:51:06.214603 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:51:06.214603 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:51:06.214603 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:51:06.214603 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:51:06.214603 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:51:06.214603 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:51:06.214603 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:51:06.214603 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 10 11:51:06.215869 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 10 11:51:06.215869 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:51:06.215869 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Oct 10 11:51:06.215869 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 653/1278] 10.222.0.46 () {56 vars in 1108 bytes} [Mon Oct 10 11:51:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:51:06.220444 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95fc617-0b1a-41a2-806e-34261473fd8d None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:51:06.327242 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95fc617-0b1a-41a2-806e-34261473fd8d None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:51:06.360580 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0beaa1cc-bc63-4910-be52-626e34e3a5ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:51:06.361192 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0beaa1cc-bc63-4910-be52-626e34e3a5ec None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:51:06.361474 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0beaa1cc-bc63-4910-be52-626e34e3a5ec None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:51:06.361752 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0beaa1cc-bc63-4910-be52-626e34e3a5ec None admin] PATH_INFO: `/v3/domains` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:51:06.362398 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0beaa1cc-bc63-4910-be52-626e34e3a5ec None admin] RBAC: Adding query filter params (name=Default) {{(pid=81676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:51:06.362726 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0beaa1cc-bc63-4910-be52-626e34e3a5ec None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:51:06.364921 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0beaa1cc-bc63-4910-be52-626e34e3a5ec None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:51:06.376633 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 626/1279] 10.222.0.46 () {56 vars in 1109 bytes} [Mon Oct 10 11:51:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:51:06.382942 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe5f0bc-34f6-4c01-b47a-33294afb6cdb None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:51:06.412323 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:51:06.484579 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe5f0bc-34f6-4c01-b47a-33294afb6cdb None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:51:06.520012 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340fb59e-4780-4987-a9d5-a47db039aa37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:51:06.520651 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340fb59e-4780-4987-a9d5-a47db039aa37 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:51:06.520935 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340fb59e-4780-4987-a9d5-a47db039aa37 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:51:06.521218 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340fb59e-4780-4987-a9d5-a47db039aa37 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:51:06.528581 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340fb59e-4780-4987-a9d5-a47db039aa37 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:51:06.530080 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340fb59e-4780-4987-a9d5-a47db039aa37 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:51:06.541461 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application [None req-340fb59e-4780-4987-a9d5-a47db039aa37 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 10 11:51:06.541461 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:51:06.541461 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:51:06.541461 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:51:06.541461 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:51:06.541461 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:51:06.541461 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:51:06.541461 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:51:06.541461 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:51:06.541461 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:51:06.541461 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:51:06.541461 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:51:06.541461 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 10 11:51:06.541461 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 10 11:51:06.541461 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 10 11:51:06.541461 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 10 11:51:06.542524 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:51:06.542524 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:51:06.542524 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 10 11:51:06.542524 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 10 11:51:06.542524 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 10 11:51:06.542524 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 10 11:51:06.542524 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:51:06.542524 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:51:06.542524 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:51:06.542524 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:51:06.542524 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:51:06.542524 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application with Lock( Oct 10 11:51:06.542524 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:51:06.542524 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:51:06.542524 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:51:06.542524 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:51:06.542524 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:51:06.542524 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:51:06.543529 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:51:06.543529 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:51:06.543529 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 10 11:51:06.543529 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 10 11:51:06.543529 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 10 11:51:06.543529 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 10 11:51:06.543529 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 10 11:51:06.543529 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 10 11:51:06.543529 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 10 11:51:06.543529 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Oct 10 11:51:06.543529 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 654/1280] 10.222.0.46 () {56 vars in 1108 bytes} [Mon Oct 10 11:51:06 2022] GET /identity/v3/users/placement => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:51:06.548999 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0beaa1cc-bc63-4910-be52-626e34e3a5ec None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:51:06.553701 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:51:06.651358 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0beaa1cc-bc63-4910-be52-626e34e3a5ec None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:51:06.690300 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fa427c-369c-4cd5-a2e5-2525ddea1015 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:51:06.690938 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fa427c-369c-4cd5-a2e5-2525ddea1015 None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:51:06.691860 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fa427c-369c-4cd5-a2e5-2525ddea1015 None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:51:06.692199 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fa427c-369c-4cd5-a2e5-2525ddea1015 None admin] PATH_INFO: `/v3/users` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:51:06.692936 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99fa427c-369c-4cd5-a2e5-2525ddea1015 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:51:06.693241 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99fa427c-369c-4cd5-a2e5-2525ddea1015 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:51:06.694684 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99fa427c-369c-4cd5-a2e5-2525ddea1015 None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:51:06.712057 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 627/1281] 10.222.0.46 () {56 vars in 1143 bytes} [Mon Oct 10 11:51:06 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:51:06.717985 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340fb59e-4780-4987-a9d5-a47db039aa37 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:51:06.808195 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:51:06.817684 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340fb59e-4780-4987-a9d5-a47db039aa37 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:51:06.850929 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313930e4-c7be-4468-aac5-e24c6826f51e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:51:06.851534 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313930e4-c7be-4468-aac5-e24c6826f51e None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:51:06.851819 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313930e4-c7be-4468-aac5-e24c6826f51e None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:51:06.852096 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313930e4-c7be-4468-aac5-e24c6826f51e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:51:06.918943 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313930e4-c7be-4468-aac5-e24c6826f51e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:51:06.920325 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313930e4-c7be-4468-aac5-e24c6826f51e None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:51:07.008009 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:51:07.017803 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 655/1282] 10.222.0.46 () {58 vars in 1282 bytes} [Mon Oct 10 11:51:06 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 10 11:51:07.023862 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fa427c-369c-4cd5-a2e5-2525ddea1015 None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:51:07.084636 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:51:07.122906 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fa427c-369c-4cd5-a2e5-2525ddea1015 None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:51:07.155725 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bb343c-1979-4990-8460-89561c4cfb07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:51:07.156360 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bb343c-1979-4990-8460-89561c4cfb07 None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:51:07.156625 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bb343c-1979-4990-8460-89561c4cfb07 None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:51:07.156898 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bb343c-1979-4990-8460-89561c4cfb07 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:51:07.167545 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15bb343c-1979-4990-8460-89561c4cfb07 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:51:07.169012 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15bb343c-1979-4990-8460-89561c4cfb07 None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:51:07.183046 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application [None req-15bb343c-1979-4990-8460-89561c4cfb07 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:51:07.183046 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:51:07.183046 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:51:07.183046 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:51:07.183046 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:51:07.183046 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:51:07.183046 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:51:07.183046 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:51:07.183046 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:51:07.183046 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:51:07.183046 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:51:07.183046 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:51:07.183046 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 10 11:51:07.183046 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 10 11:51:07.183046 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 10 11:51:07.183046 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 10 11:51:07.186927 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:51:07.186927 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:51:07.186927 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:51:07.186927 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:51:07.186927 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:51:07.186927 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:51:07.186927 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:51:07.186927 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application with Lock( Oct 10 11:51:07.186927 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:51:07.186927 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:51:07.186927 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:51:07.186927 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:51:07.186927 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:51:07.186927 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:51:07.186927 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:51:07.186927 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:51:07.186927 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 10 11:51:07.187821 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 10 11:51:07.187821 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 10 11:51:07.187821 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Oct 10 11:51:07.187821 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 628/1283] 10.222.0.46 () {56 vars in 1108 bytes} [Mon Oct 10 11:51:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:51:07.190117 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313930e4-c7be-4468-aac5-e24c6826f51e None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:51:07.291039 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313930e4-c7be-4468-aac5-e24c6826f51e None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:51:07.323710 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82aa609a-102a-4c9d-ba3a-cf5a176caa87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:51:07.324317 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82aa609a-102a-4c9d-ba3a-cf5a176caa87 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:51:07.324601 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82aa609a-102a-4c9d-ba3a-cf5a176caa87 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:51:07.324879 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82aa609a-102a-4c9d-ba3a-cf5a176caa87 None admin] PATH_INFO: `/v3/domains` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:51:07.325511 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82aa609a-102a-4c9d-ba3a-cf5a176caa87 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:51:07.325816 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82aa609a-102a-4c9d-ba3a-cf5a176caa87 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:51:07.327218 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82aa609a-102a-4c9d-ba3a-cf5a176caa87 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:51:07.339169 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 656/1284] 10.222.0.46 () {56 vars in 1109 bytes} [Mon Oct 10 11:51:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:51:07.345033 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bb343c-1979-4990-8460-89561c4cfb07 None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:51:07.444921 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bb343c-1979-4990-8460-89561c4cfb07 None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:51:07.461090 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:51:07.494812 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d83d8f-04ed-4100-9276-da78381322d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:51:07.494812 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d83d8f-04ed-4100-9276-da78381322d5 None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:51:07.494812 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d83d8f-04ed-4100-9276-da78381322d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:51:07.494812 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d83d8f-04ed-4100-9276-da78381322d5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:51:07.495188 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d83d8f-04ed-4100-9276-da78381322d5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:51:07.495188 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d83d8f-04ed-4100-9276-da78381322d5 None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:51:07.502262 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application [None req-47d83d8f-04ed-4100-9276-da78381322d5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 10 11:51:07.502262 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:51:07.502262 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:51:07.502262 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:51:07.502262 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:51:07.502262 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:51:07.502262 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:51:07.502262 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:51:07.502262 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:51:07.502262 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:51:07.502262 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:51:07.502262 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:51:07.502262 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 10 11:51:07.502262 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 10 11:51:07.502262 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 10 11:51:07.503289 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 10 11:51:07.503289 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:51:07.503289 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:51:07.503289 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:51:07.503289 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:51:07.503289 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:51:07.503289 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:51:07.503289 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:51:07.503289 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application with Lock( Oct 10 11:51:07.503289 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:51:07.503289 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:51:07.503289 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:51:07.503289 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:51:07.503289 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:51:07.503289 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:51:07.503289 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:51:07.503289 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:51:07.504992 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 10 11:51:07.504992 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 10 11:51:07.504992 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 10 11:51:07.504992 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 10 11:51:07.504992 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 10 11:51:07.504992 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 10 11:51:07.504992 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 10 11:51:07.504992 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: ERROR keystone.server.flask.application Oct 10 11:51:07.504992 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 629/1285] 10.222.0.46 () {56 vars in 1111 bytes} [Mon Oct 10 11:51:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:51:07.509827 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82aa609a-102a-4c9d-ba3a-cf5a176caa87 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:51:07.559690 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:51:07.608397 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82aa609a-102a-4c9d-ba3a-cf5a176caa87 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:51:07.642306 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c033a8-5592-4fdd-8349-253dadab5413 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:51:07.642948 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c033a8-5592-4fdd-8349-253dadab5413 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:51:07.643226 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c033a8-5592-4fdd-8349-253dadab5413 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:51:07.643510 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c033a8-5592-4fdd-8349-253dadab5413 None admin] PATH_INFO: `/v3/projects` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:51:07.644178 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c033a8-5592-4fdd-8349-253dadab5413 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:51:07.645030 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c033a8-5592-4fdd-8349-253dadab5413 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:51:07.646502 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c033a8-5592-4fdd-8349-253dadab5413 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:51:07.659405 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 657/1286] 10.222.0.46 () {56 vars in 1148 bytes} [Mon Oct 10 11:51:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:51:07.665615 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d83d8f-04ed-4100-9276-da78381322d5 None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:51:07.766409 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d83d8f-04ed-4100-9276-da78381322d5 None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:51:07.802640 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817e90b1-a28f-4a44-a5b5-a0e3509e6df6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:51:07.803276 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817e90b1-a28f-4a44-a5b5-a0e3509e6df6 None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:51:07.803629 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817e90b1-a28f-4a44-a5b5-a0e3509e6df6 None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:51:07.803909 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817e90b1-a28f-4a44-a5b5-a0e3509e6df6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:51:07.804633 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817e90b1-a28f-4a44-a5b5-a0e3509e6df6 None admin] RBAC: Adding query filter params (role.id=0d822a0674b6495fac623877a3b38025, scope.project.id=4ad3772f7b6a47c28b7ba37a9316a3c1, user.id=8e8f9bb1776d47169ef7febb57c954cf) {{(pid=81676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:51:07.804945 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817e90b1-a28f-4a44-a5b5-a0e3509e6df6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:51:07.806382 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817e90b1-a28f-4a44-a5b5-a0e3509e6df6 None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:51:07.812147 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 630/1287] 10.222.0.46 () {56 vars in 1374 bytes} [Mon Oct 10 11:51:07 2022] GET /identity/v3/role_assignments?user.id=8e8f9bb1776d47169ef7febb57c954cf&scope.project.id=4ad3772f7b6a47c28b7ba37a9316a3c1&role.id=0d822a0674b6495fac623877a3b38025 => generated 608 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:51:10.027435 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6955ded9-f584-4151-84f0-365922312639 None None] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:51:10.027973 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6955ded9-f584-4151-84f0-365922312639 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:51:10.027973 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6955ded9-f584-4151-84f0-365922312639 None None] PATH_INFO: `/` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:51:10.029331 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 658/1288] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 10 11:51:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:51:10.034844 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76050824-ed34-46ae-b3e4-4a4e8c0f0b65 None None] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:51:10.035791 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76050824-ed34-46ae-b3e4-4a4e8c0f0b65 None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:51:10.035905 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76050824-ed34-46ae-b3e4-4a4e8c0f0b65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:51:10.054785 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.auth.core [None req-76050824-ed34-46ae-b3e4-4a4e8c0f0b65 None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:51:10.056116 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-76050824-ed34-46ae-b3e4-4a4e8c0f0b65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:51:10.085660 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:51:10.123163 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 631/1289] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 10 11:51:10 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 10 11:51:10.399906 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6955ded9-f584-4151-84f0-365922312639 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:51:10.504474 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6955ded9-f584-4151-84f0-365922312639 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:51:10.514931 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:51:10.537778 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64892929-9aee-44ca-910a-5a0c2dd0617b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:51:10.538396 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64892929-9aee-44ca-910a-5a0c2dd0617b None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:51:10.540871 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64892929-9aee-44ca-910a-5a0c2dd0617b None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:51:10.540871 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64892929-9aee-44ca-910a-5a0c2dd0617b None admin] PATH_INFO: `/v3/services/placement` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:51:10.540871 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64892929-9aee-44ca-910a-5a0c2dd0617b None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:51:10.540871 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64892929-9aee-44ca-910a-5a0c2dd0617b None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:51:10.549683 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application [None req-64892929-9aee-44ca-910a-5a0c2dd0617b None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Oct 10 11:51:10.549683 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:51:10.549683 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 10 11:51:10.549683 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 10 11:51:10.549683 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 10 11:51:10.549683 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 10 11:51:10.549683 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 10 11:51:10.549683 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 10 11:51:10.549683 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 10 11:51:10.549683 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Oct 10 11:51:10.549683 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 10 11:51:10.549683 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Oct 10 11:51:10.549683 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:51:10.549683 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:51:10.549683 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:51:10.549683 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:51:10.549683 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:51:10.550919 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:51:10.550919 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:51:10.550919 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:51:10.550919 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:51:10.550919 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:51:10.550919 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:51:10.550919 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 10 11:51:10.550919 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 10 11:51:10.550919 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 10 11:51:10.550919 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 10 11:51:10.550919 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:51:10.550919 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:51:10.550919 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:51:10.550919 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:51:10.550919 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:51:10.550919 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:51:10.551818 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:51:10.551818 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application with Lock( Oct 10 11:51:10.551818 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:51:10.551818 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:51:10.551818 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:51:10.551818 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:51:10.551818 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:51:10.551818 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:51:10.551818 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:51:10.551818 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:51:10.551818 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 10 11:51:10.551818 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 10 11:51:10.551818 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 10 11:51:10.551818 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Oct 10 11:51:10.551818 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 659/1290] 10.222.0.46 () {56 vars in 1117 bytes} [Mon Oct 10 11:51:10 2022] GET /identity/v3/services/placement => generated 90 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:51:10.557344 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76050824-ed34-46ae-b3e4-4a4e8c0f0b65 None None] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:51:10.653768 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76050824-ed34-46ae-b3e4-4a4e8c0f0b65 None None] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:51:10.686610 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7694aa-aef6-4b13-af58-1e96e87fec0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:51:10.687305 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7694aa-aef6-4b13-af58-1e96e87fec0a None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:51:10.687586 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7694aa-aef6-4b13-af58-1e96e87fec0a None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:51:10.687859 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7694aa-aef6-4b13-af58-1e96e87fec0a None admin] PATH_INFO: `/v3/services` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:51:10.688485 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7694aa-aef6-4b13-af58-1e96e87fec0a None admin] RBAC: Adding query filter params (name=placement) {{(pid=81676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:51:10.688789 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7694aa-aef6-4b13-af58-1e96e87fec0a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:51:10.690149 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7694aa-aef6-4b13-af58-1e96e87fec0a None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:51:10.695442 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 632/1291] 10.222.0.46 () {56 vars in 1116 bytes} [Mon Oct 10 11:51:10 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:51:10.701207 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64892929-9aee-44ca-910a-5a0c2dd0617b None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:51:10.801063 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64892929-9aee-44ca-910a-5a0c2dd0617b None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:51:10.835172 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579f2df5-862d-428b-8631-a3dabf60de3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:51:10.835784 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579f2df5-862d-428b-8631-a3dabf60de3f None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:51:10.836068 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579f2df5-862d-428b-8631-a3dabf60de3f None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:51:10.836347 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579f2df5-862d-428b-8631-a3dabf60de3f None admin] PATH_INFO: `/v3/services` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:51:10.836974 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579f2df5-862d-428b-8631-a3dabf60de3f None admin] RBAC: Adding query filter params (type=placement) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:51:10.837282 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579f2df5-862d-428b-8631-a3dabf60de3f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:51:10.838634 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579f2df5-862d-428b-8631-a3dabf60de3f None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:51:10.842983 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 660/1292] 10.222.0.46 () {56 vars in 1116 bytes} [Mon Oct 10 11:51:10 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:51:13.008829 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51b36fa-d253-4d9f-a293-0d89aad69794 None None] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:51:13.009423 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51b36fa-d253-4d9f-a293-0d89aad69794 None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:51:13.009423 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51b36fa-d253-4d9f-a293-0d89aad69794 None None] PATH_INFO: `/` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:51:13.010732 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 633/1293] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 10 11:51:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:51:13.015956 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303423dc-3e00-45c1-9f4b-971d87d0e47e None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:51:13.016418 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303423dc-3e00-45c1-9f4b-971d87d0e47e None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:51:13.016527 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303423dc-3e00-45c1-9f4b-971d87d0e47e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:51:13.034828 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-303423dc-3e00-45c1-9f4b-971d87d0e47e None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:51:13.036502 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-303423dc-3e00-45c1-9f4b-971d87d0e47e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:51:13.065392 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:51:13.103524 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 661/1294] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 10 11:51:13 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 10 11:51:13.379366 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51b36fa-d253-4d9f-a293-0d89aad69794 None None] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:51:13.410908 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:51:13.484394 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51b36fa-d253-4d9f-a293-0d89aad69794 None None] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:51:13.520084 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43abfc69-3b9c-45fc-b311-53776b9bcb9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:51:13.520682 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43abfc69-3b9c-45fc-b311-53776b9bcb9f None admin] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:51:13.521003 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43abfc69-3b9c-45fc-b311-53776b9bcb9f None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:51:13.521279 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43abfc69-3b9c-45fc-b311-53776b9bcb9f None admin] PATH_INFO: `/v3/services` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:51:13.521976 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43abfc69-3b9c-45fc-b311-53776b9bcb9f None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:51:13.523359 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43abfc69-3b9c-45fc-b311-53776b9bcb9f None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:51:13.668646 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 634/1295] 10.222.0.46 () {60 vars in 1141 bytes} [Mon Oct 10 11:51:13 2022] POST /identity/v3/services => generated 246 bytes in 290 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 11:51:15.821825 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d5468a-0c4c-4331-9f40-97fd7af76eeb None None] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:51:15.822329 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d5468a-0c4c-4331-9f40-97fd7af76eeb None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:51:15.822329 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d5468a-0c4c-4331-9f40-97fd7af76eeb None None] PATH_INFO: `/` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:51:15.823665 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 662/1296] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 10 11:51:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:51:15.828859 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f441e3e-1da6-4788-b30f-c648272da6fe None None] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:51:15.829243 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f441e3e-1da6-4788-b30f-c648272da6fe None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:51:15.829409 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f441e3e-1da6-4788-b30f-c648272da6fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:51:15.848086 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.auth.core [None req-9f441e3e-1da6-4788-b30f-c648272da6fe None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:51:15.849360 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-9f441e3e-1da6-4788-b30f-c648272da6fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:51:15.922934 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 635/1297] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 10 11:51:15 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 10 11:51:16.198381 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d5468a-0c4c-4331-9f40-97fd7af76eeb None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:51:16.305635 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d5468a-0c4c-4331-9f40-97fd7af76eeb None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:51:16.340685 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5351ac0c-c19a-4e71-96f0-bdb03cd400b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:51:16.341324 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5351ac0c-c19a-4e71-96f0-bdb03cd400b1 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:51:16.341603 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5351ac0c-c19a-4e71-96f0-bdb03cd400b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:51:16.341875 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5351ac0c-c19a-4e71-96f0-bdb03cd400b1 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:51:16.342459 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5351ac0c-c19a-4e71-96f0-bdb03cd400b1 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:51:16.343852 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5351ac0c-c19a-4e71-96f0-bdb03cd400b1 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:51:16.353175 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application [None req-5351ac0c-c19a-4e71-96f0-bdb03cd400b1 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Oct 10 11:51:16.353175 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:51:16.353175 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 10 11:51:16.353175 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 10 11:51:16.353175 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 10 11:51:16.353175 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 10 11:51:16.353175 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 10 11:51:16.353175 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 10 11:51:16.353175 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 10 11:51:16.353175 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Oct 10 11:51:16.353175 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 10 11:51:16.353175 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Oct 10 11:51:16.353175 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:51:16.353175 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:51:16.353175 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:51:16.353175 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:51:16.353175 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:51:16.354276 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:51:16.354276 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:51:16.354276 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:51:16.354276 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:51:16.354276 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:51:16.354276 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:51:16.354276 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 10 11:51:16.354276 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 10 11:51:16.354276 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 10 11:51:16.354276 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 10 11:51:16.354276 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:51:16.354276 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:51:16.354276 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:51:16.354276 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:51:16.354276 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:51:16.354276 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:51:16.355137 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:51:16.355137 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application with Lock( Oct 10 11:51:16.355137 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:51:16.355137 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:51:16.355137 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:51:16.355137 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:51:16.355137 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:51:16.355137 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:51:16.355137 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:51:16.355137 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:51:16.355137 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 10 11:51:16.355137 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 10 11:51:16.355137 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 10 11:51:16.355137 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Oct 10 11:51:16.355137 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 663/1298] 10.222.0.46 () {56 vars in 1117 bytes} [Mon Oct 10 11:51:16 2022] GET /identity/v3/services/placement => generated 90 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:51:16.360379 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f441e3e-1da6-4788-b30f-c648272da6fe None None] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:51:16.378641 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:51:16.458276 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f441e3e-1da6-4788-b30f-c648272da6fe None None] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:51:16.492865 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e39e7ae-b6d8-4651-95e7-cccaca660913 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:51:16.493484 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e39e7ae-b6d8-4651-95e7-cccaca660913 None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:51:16.500828 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e39e7ae-b6d8-4651-95e7-cccaca660913 None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:51:16.500828 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e39e7ae-b6d8-4651-95e7-cccaca660913 None admin] PATH_INFO: `/v3/services` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:51:16.500828 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e39e7ae-b6d8-4651-95e7-cccaca660913 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:51:16.500828 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e39e7ae-b6d8-4651-95e7-cccaca660913 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:51:16.500828 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e39e7ae-b6d8-4651-95e7-cccaca660913 None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:51:16.501609 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 636/1299] 10.222.0.46 () {56 vars in 1116 bytes} [Mon Oct 10 11:51:16 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 11:51:16.508373 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5351ac0c-c19a-4e71-96f0-bdb03cd400b1 None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:51:16.540505 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:51:16.605902 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5351ac0c-c19a-4e71-96f0-bdb03cd400b1 None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:51:16.638513 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e259ec-286b-444d-ba63-0daaa41107a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:51:16.639133 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e259ec-286b-444d-ba63-0daaa41107a9 None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:51:16.639402 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e259ec-286b-444d-ba63-0daaa41107a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:51:16.639673 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e259ec-286b-444d-ba63-0daaa41107a9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:51:16.640353 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e259ec-286b-444d-ba63-0daaa41107a9 None admin] RBAC: Adding query filter params (interface=public, service_id=f2bcf85b08ce448286c85b4e054d4c9d, region_id=RegionOne) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:51:16.640653 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e259ec-286b-444d-ba63-0daaa41107a9 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:51:16.642024 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e259ec-286b-444d-ba63-0daaa41107a9 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:51:16.647719 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 664/1300] 10.222.0.46 () {56 vars in 1251 bytes} [Mon Oct 10 11:51:16 2022] GET /identity/v3/endpoints?service_id=f2bcf85b08ce448286c85b4e054d4c9d&interface=public®ion_id=RegionOne => generated 194 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:51:16.653549 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e39e7ae-b6d8-4651-95e7-cccaca660913 None admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:51:16.732765 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:51:16.751425 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e39e7ae-b6d8-4651-95e7-cccaca660913 None admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:51:16.785056 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa4b1a8-1d12-49d7-aaab-91c68e1b328a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:51:16.785671 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa4b1a8-1d12-49d7-aaab-91c68e1b328a None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:51:16.785952 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa4b1a8-1d12-49d7-aaab-91c68e1b328a None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:51:16.786227 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa4b1a8-1d12-49d7-aaab-91c68e1b328a None admin] PATH_INFO: `/v3/services` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:51:16.786866 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa4b1a8-1d12-49d7-aaab-91c68e1b328a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:51:16.788229 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa4b1a8-1d12-49d7-aaab-91c68e1b328a None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:51:16.795128 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 637/1301] 10.222.0.46 () {56 vars in 1087 bytes} [Mon Oct 10 11:51:16 2022] GET /identity/v3/services => generated 2191 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 11:51:18.973889 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7bb777-57f8-4dfe-beee-5d99e4f08fb4 None None] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:51:18.974422 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7bb777-57f8-4dfe-beee-5d99e4f08fb4 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:51:18.974422 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7bb777-57f8-4dfe-beee-5d99e4f08fb4 None None] PATH_INFO: `/` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:51:18.975838 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 665/1302] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 10 11:51:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:51:18.981070 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4b597f-dc01-43e8-8886-f546da3385c3 None None] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:51:18.981494 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4b597f-dc01-43e8-8886-f546da3385c3 None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:51:18.988910 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4b597f-dc01-43e8-8886-f546da3385c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:51:19.001120 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.auth.core [None req-eb4b597f-dc01-43e8-8886-f546da3385c3 None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:51:19.002302 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-eb4b597f-dc01-43e8-8886-f546da3385c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:51:19.068469 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 638/1303] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 10 11:51:18 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 10 11:51:19.344530 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7bb777-57f8-4dfe-beee-5d99e4f08fb4 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:51:19.438890 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:51:19.446216 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7bb777-57f8-4dfe-beee-5d99e4f08fb4 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:51:19.480851 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42975fbb-1f55-4570-bd6a-14d8b255fb1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:51:19.481598 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42975fbb-1f55-4570-bd6a-14d8b255fb1c None admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:51:19.481742 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42975fbb-1f55-4570-bd6a-14d8b255fb1c None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:51:19.482012 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42975fbb-1f55-4570-bd6a-14d8b255fb1c None admin] PATH_INFO: `/v3/services/placement` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:51:19.482583 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42975fbb-1f55-4570-bd6a-14d8b255fb1c None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:51:19.483966 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42975fbb-1f55-4570-bd6a-14d8b255fb1c None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:51:19.493576 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application [None req-42975fbb-1f55-4570-bd6a-14d8b255fb1c None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Oct 10 11:51:19.493576 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:51:19.493576 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 10 11:51:19.493576 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 10 11:51:19.493576 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 10 11:51:19.493576 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 10 11:51:19.493576 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 10 11:51:19.493576 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 10 11:51:19.493576 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 10 11:51:19.493576 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Oct 10 11:51:19.493576 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 10 11:51:19.493576 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Oct 10 11:51:19.493576 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 10 11:51:19.493576 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 10 11:51:19.493576 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 10 11:51:19.493576 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 10 11:51:19.493576 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 10 11:51:19.494930 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 10 11:51:19.494930 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 10 11:51:19.494930 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 10 11:51:19.494930 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 10 11:51:19.494930 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 10 11:51:19.494930 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 10 11:51:19.494930 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 10 11:51:19.494930 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 10 11:51:19.494930 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 10 11:51:19.494930 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 10 11:51:19.494930 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 10 11:51:19.494930 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 10 11:51:19.494930 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 10 11:51:19.494930 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 10 11:51:19.494930 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 10 11:51:19.494930 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.get_or_create( Oct 10 11:51:19.496037 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 10 11:51:19.496037 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application with Lock( Oct 10 11:51:19.496037 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 10 11:51:19.496037 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self._enter() Oct 10 11:51:19.496037 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 10 11:51:19.496037 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 10 11:51:19.496037 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 10 11:51:19.496037 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application return self.creator() Oct 10 11:51:19.496037 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 10 11:51:19.496037 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application created_value = creator( Oct 10 11:51:19.496037 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 10 11:51:19.496037 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 10 11:51:19.496037 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 10 11:51:19.496037 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: ERROR keystone.server.flask.application Oct 10 11:51:19.496037 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 666/1304] 10.222.0.46 () {56 vars in 1117 bytes} [Mon Oct 10 11:51:19 2022] GET /identity/v3/services/placement => generated 90 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 10 11:51:19.500712 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4b597f-dc01-43e8-8886-f546da3385c3 None None] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:51:19.582528 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:51:19.604264 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4b597f-dc01-43e8-8886-f546da3385c3 None None] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:51:19.639104 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8784686b-7f68-47d9-8847-9c59253ea290 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:51:19.639712 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8784686b-7f68-47d9-8847-9c59253ea290 None admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:51:19.639988 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8784686b-7f68-47d9-8847-9c59253ea290 None admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:51:19.640258 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8784686b-7f68-47d9-8847-9c59253ea290 None admin] PATH_INFO: `/v3/services` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:51:19.640884 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8784686b-7f68-47d9-8847-9c59253ea290 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 11:51:19.641180 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8784686b-7f68-47d9-8847-9c59253ea290 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:51:19.642541 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8784686b-7f68-47d9-8847-9c59253ea290 None admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:51:19.647722 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 639/1305] 10.222.0.46 () {56 vars in 1116 bytes} [Mon Oct 10 11:51:19 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 11:51:19.653701 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42975fbb-1f55-4570-bd6a-14d8b255fb1c None admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:51:19.759201 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42975fbb-1f55-4570-bd6a-14d8b255fb1c None admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:51:19.793417 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10ece71-55ad-4a76-bec5-e9b425eed956 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:51:19.794032 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10ece71-55ad-4a76-bec5-e9b425eed956 None admin] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:51:19.794313 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10ece71-55ad-4a76-bec5-e9b425eed956 None admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:51:19.794587 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10ece71-55ad-4a76-bec5-e9b425eed956 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:51:19.795298 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10ece71-55ad-4a76-bec5-e9b425eed956 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:51:19.796666 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10ece71-55ad-4a76-bec5-e9b425eed956 None admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:51:19.913100 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 667/1306] 10.222.0.46 () {60 vars in 1144 bytes} [Mon Oct 10 11:51:19 2022] POST /identity/v3/endpoints => generated 331 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 11:51:29.644645 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=30 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:51:29.907622 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:59:03.286534 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ecec18-4fe8-488a-a224-b83a8625620b None None] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:59:03.294604 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ecec18-4fe8-488a-a224-b83a8625620b None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:59:03.294604 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ecec18-4fe8-488a-a224-b83a8625620b None None] PATH_INFO: `/` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:59:03.294604 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 640/1307] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 10 11:59:03 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:59:03.296921 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23727b99-c314-46db-aca2-b3f32e00e5ac None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:59:03.297449 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23727b99-c314-46db-aca2-b3f32e00e5ac None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:59:03.297885 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23727b99-c314-46db-aca2-b3f32e00e5ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:59:03.447248 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-23727b99-c314-46db-aca2-b3f32e00e5ac None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:59:03.448868 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-23727b99-c314-46db-aca2-b3f32e00e5ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:59:03.583025 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 668/1308] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 10 11:59:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 288 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 11:59:03.882591 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdee620-6956-480c-a4a8-333b0248d7fa None None] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:59:03.882939 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdee620-6956-480c-a4a8-333b0248d7fa None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:59:03.883168 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdee620-6956-480c-a4a8-333b0248d7fa None None] PATH_INFO: `/` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:59:03.884635 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 641/1309] 10.222.0.46 () {54 vars in 964 bytes} [Mon Oct 10 11:59:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:59:03.890739 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c8d085-ef95-4494-8cfb-b2aee1f51535 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:59:03.891114 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c8d085-ef95-4494-8cfb-b2aee1f51535 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:59:03.891465 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c8d085-ef95-4494-8cfb-b2aee1f51535 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:59:03.995212 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-f6c8d085-ef95-4494-8cfb-b2aee1f51535 None None] MFA Rules not processed for user `58b1c7c85302421cba823ace4201861f`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:59:03.996437 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-f6c8d085-ef95-4494-8cfb-b2aee1f51535 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:59:04.022030 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:59:04.099305 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 669/1310] 10.222.0.46 () {58 vars in 1061 bytes} [Mon Oct 10 11:59:03 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 11:59:04.108348 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdee620-6956-480c-a4a8-333b0248d7fa None None] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:59:04.110211 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-ffdee620-6956-480c-a4a8-333b0248d7fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:59:04.223309 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdee620-6956-480c-a4a8-333b0248d7fa None None] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:59:04.305815 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629ca36d-5b5e-45ad-bf54-396a7eaab1d1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58b1c7c85302421cba823ace4201861f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad3772f7b6a47c28b7ba37a9316a3c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:59:04.306540 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629ca36d-5b5e-45ad-bf54-396a7eaab1d1 service neutron] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:59:04.306947 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629ca36d-5b5e-45ad-bf54-396a7eaab1d1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:59:04.307350 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629ca36d-5b5e-45ad-bf54-396a7eaab1d1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:59:04.309096 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-629ca36d-5b5e-45ad-bf54-396a7eaab1d1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:59:04.375112 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:59:04.449869 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629ca36d-5b5e-45ad-bf54-396a7eaab1d1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:59:04.452622 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629ca36d-5b5e-45ad-bf54-396a7eaab1d1 service neutron] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:59:04.484519 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 642/1311] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 10 11:59:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 11:59:06.981898 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba61e9b-048f-46c8-a6a6-d9a22099b665 None None] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:59:06.982465 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba61e9b-048f-46c8-a6a6-d9a22099b665 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:59:06.982465 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba61e9b-048f-46c8-a6a6-d9a22099b665 None None] PATH_INFO: `/` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:59:06.983829 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 670/1312] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 10 11:59:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:59:06.989877 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc31a098-ae57-4d75-b03f-e469f699dc90 None None] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:59:06.990392 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc31a098-ae57-4d75-b03f-e469f699dc90 None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:59:06.990554 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc31a098-ae57-4d75-b03f-e469f699dc90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:59:07.069867 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.auth.core [None req-dc31a098-ae57-4d75-b03f-e469f699dc90 None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:59:07.071163 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-dc31a098-ae57-4d75-b03f-e469f699dc90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:59:07.183174 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 643/1313] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 10 11:59:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 11:59:07.478482 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba61e9b-048f-46c8-a6a6-d9a22099b665 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:59:07.480078 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-7ba61e9b-048f-46c8-a6a6-d9a22099b665 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:59:07.592959 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba61e9b-048f-46c8-a6a6-d9a22099b665 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:59:07.670802 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:59:07.673191 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d7dfee-f110-4897-b2a3-0446b26251b6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58b1c7c85302421cba823ace4201861f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad3772f7b6a47c28b7ba37a9316a3c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:59:07.673864 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d7dfee-f110-4897-b2a3-0446b26251b6 service neutron] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:59:07.674136 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d7dfee-f110-4897-b2a3-0446b26251b6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:59:07.674402 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d7dfee-f110-4897-b2a3-0446b26251b6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:59:07.676039 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-87d7dfee-f110-4897-b2a3-0446b26251b6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:59:07.802613 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d7dfee-f110-4897-b2a3-0446b26251b6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:59:07.803999 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d7dfee-f110-4897-b2a3-0446b26251b6 service neutron] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:59:07.823263 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 671/1314] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 10 11:59:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 11:59:10.596843 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed440f11-6664-4102-bb07-59656db6f959 None None] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:59:10.597422 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed440f11-6664-4102-bb07-59656db6f959 None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:59:10.597422 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed440f11-6664-4102-bb07-59656db6f959 None None] PATH_INFO: `/` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:59:10.598772 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 644/1315] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 10 11:59:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:59:10.603911 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9e4de3-0666-4d58-804a-11388b9c195d None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:59:10.604292 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9e4de3-0666-4d58-804a-11388b9c195d None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:59:10.604470 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9e4de3-0666-4d58-804a-11388b9c195d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:59:10.671227 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-0a9e4de3-0666-4d58-804a-11388b9c195d None None] MFA Rules not processed for user `531d202e4dea4ee38988bfd8b9e26751`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:59:10.672457 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-0a9e4de3-0666-4d58-804a-11388b9c195d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:59:10.723977 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:59:10.769081 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 672/1316] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 10 11:59:10 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 11:59:11.055808 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed440f11-6664-4102-bb07-59656db6f959 None None] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:59:11.057365 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-ed440f11-6664-4102-bb07-59656db6f959 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:59:11.067280 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:59:11.164877 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed440f11-6664-4102-bb07-59656db6f959 None None] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:59:11.237895 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d842f2d2-acd6-4939-96c1-098882068617 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58b1c7c85302421cba823ace4201861f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad3772f7b6a47c28b7ba37a9316a3c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:59:11.238501 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d842f2d2-acd6-4939-96c1-098882068617 service neutron] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:59:11.238803 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d842f2d2-acd6-4939-96c1-098882068617 service neutron] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:59:11.239330 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d842f2d2-acd6-4939-96c1-098882068617 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:59:11.240924 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-d842f2d2-acd6-4939-96c1-098882068617 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:59:11.303709 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:59:11.351144 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d842f2d2-acd6-4939-96c1-098882068617 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:59:11.352661 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d842f2d2-acd6-4939-96c1-098882068617 service neutron] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:59:11.376253 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 645/1317] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 10 11:59:11 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 11:59:14.282369 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab439a78-c3d3-4c15-b86e-5a9045186a92 None None] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:59:14.282972 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab439a78-c3d3-4c15-b86e-5a9045186a92 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:59:14.282972 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab439a78-c3d3-4c15-b86e-5a9045186a92 None None] PATH_INFO: `/` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:59:14.284317 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 673/1318] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 10 11:59:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 10 11:59:14.289565 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91b06db-81a2-4f58-bf49-517b0aefbdf1 None None] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:59:14.289985 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91b06db-81a2-4f58-bf49-517b0aefbdf1 None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:59:14.290100 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91b06db-81a2-4f58-bf49-517b0aefbdf1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:59:14.355929 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.auth.core [None req-d91b06db-81a2-4f58-bf49-517b0aefbdf1 None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:59:14.357139 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-d91b06db-81a2-4f58-bf49-517b0aefbdf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:59:14.453343 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 646/1319] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 10 11:59:14 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 11:59:14.747393 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e291b1-d971-4d24-8dc1-502d1237491b None None] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:59:14.747678 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e291b1-d971-4d24-8dc1-502d1237491b None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:59:14.748022 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e291b1-d971-4d24-8dc1-502d1237491b None None] PATH_INFO: `/` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:59:14.749324 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 674/1320] 10.222.0.46 () {54 vars in 964 bytes} [Mon Oct 10 11:59:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:59:14.755421 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564d6231-ce7f-4a23-bf55-2ef97cb7f36f None None] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:59:14.755709 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564d6231-ce7f-4a23-bf55-2ef97cb7f36f None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:59:14.756007 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564d6231-ce7f-4a23-bf55-2ef97cb7f36f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:59:14.777889 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:59:14.830824 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.auth.core [None req-564d6231-ce7f-4a23-bf55-2ef97cb7f36f None None] MFA Rules not processed for user `58b1c7c85302421cba823ace4201861f`. Rule list: `[]` (Enabled: `True`). {{(pid=81676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:59:14.832022 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-564d6231-ce7f-4a23-bf55-2ef97cb7f36f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:59:14.910553 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 647/1321] 10.222.0.46 () {58 vars in 1061 bytes} [Mon Oct 10 11:59:14 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 11:59:14.920880 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e291b1-d971-4d24-8dc1-502d1237491b None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:59:14.922503 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-18e291b1-d971-4d24-8dc1-502d1237491b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:59:15.041376 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e291b1-d971-4d24-8dc1-502d1237491b None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:59:15.116272 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d508bdd7-3d81-452b-98ea-cf48962d923c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58b1c7c85302421cba823ace4201861f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad3772f7b6a47c28b7ba37a9316a3c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:59:15.116874 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d508bdd7-3d81-452b-98ea-cf48962d923c service neutron] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:59:15.117150 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d508bdd7-3d81-452b-98ea-cf48962d923c service neutron] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:59:15.117415 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d508bdd7-3d81-452b-98ea-cf48962d923c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:59:15.119043 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-d508bdd7-3d81-452b-98ea-cf48962d923c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:59:15.126329 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:59:15.236941 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d508bdd7-3d81-452b-98ea-cf48962d923c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:59:15.238425 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d508bdd7-3d81-452b-98ea-cf48962d923c service neutron] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:59:15.257501 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 675/1322] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 10 11:59:14 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 11:59:18.836015 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2499e3f5-fc88-4544-bc91-ccaf4f478d8d None None] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:59:18.836544 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2499e3f5-fc88-4544-bc91-ccaf4f478d8d None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:59:18.836544 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2499e3f5-fc88-4544-bc91-ccaf4f478d8d None None] PATH_INFO: `/` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:59:18.837874 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 648/1323] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 10 11:59:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:59:18.843454 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327b7d5c-02ae-42d6-8d54-1cd2c5672886 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:59:18.843923 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327b7d5c-02ae-42d6-8d54-1cd2c5672886 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:59:18.844028 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327b7d5c-02ae-42d6-8d54-1cd2c5672886 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:59:18.910954 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-327b7d5c-02ae-42d6-8d54-1cd2c5672886 None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:59:18.912189 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-327b7d5c-02ae-42d6-8d54-1cd2c5672886 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:59:18.978523 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:59:19.022178 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 676/1324] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 10 11:59:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 10 11:59:19.306706 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2499e3f5-fc88-4544-bc91-ccaf4f478d8d None None] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:59:19.308475 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-2499e3f5-fc88-4544-bc91-ccaf4f478d8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:59:19.417156 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2499e3f5-fc88-4544-bc91-ccaf4f478d8d None None] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:59:19.431019 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:59:19.498335 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7267535-758f-4909-87c4-544190e71cd0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58b1c7c85302421cba823ace4201861f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad3772f7b6a47c28b7ba37a9316a3c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:59:19.498973 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7267535-758f-4909-87c4-544190e71cd0 service neutron] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:59:19.499250 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7267535-758f-4909-87c4-544190e71cd0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:59:19.499521 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7267535-758f-4909-87c4-544190e71cd0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:59:19.501132 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-b7267535-758f-4909-87c4-544190e71cd0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:59:19.619731 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7267535-758f-4909-87c4-544190e71cd0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:59:19.621158 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7267535-758f-4909-87c4-544190e71cd0 service neutron] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:59:19.639582 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 649/1325] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 10 11:59:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 11:59:21.867521 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391d9627-4804-471c-99b2-27428770b2b7 None None] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:59:21.868051 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391d9627-4804-471c-99b2-27428770b2b7 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:59:21.868172 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391d9627-4804-471c-99b2-27428770b2b7 None None] PATH_INFO: `/` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:59:21.869517 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 677/1326] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 10 11:59:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:59:21.875519 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0800d4a5-7b2b-4f39-bdf6-9203a595e850 None None] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:59:21.875964 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0800d4a5-7b2b-4f39-bdf6-9203a595e850 None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:59:21.876059 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0800d4a5-7b2b-4f39-bdf6-9203a595e850 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:59:21.925267 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:59:21.944478 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.auth.core [None req-0800d4a5-7b2b-4f39-bdf6-9203a595e850 None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:59:21.945878 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-0800d4a5-7b2b-4f39-bdf6-9203a595e850 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:59:22.056363 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 650/1327] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 10 11:59:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 11:59:22.337664 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391d9627-4804-471c-99b2-27428770b2b7 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:59:22.339218 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-391d9627-4804-471c-99b2-27428770b2b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:59:22.450078 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391d9627-4804-471c-99b2-27428770b2b7 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:59:22.539302 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28dcc6ca-bfc7-4e5a-b201-6bacd53b4b99 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58b1c7c85302421cba823ace4201861f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad3772f7b6a47c28b7ba37a9316a3c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:59:22.539920 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28dcc6ca-bfc7-4e5a-b201-6bacd53b4b99 service neutron] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:59:22.540198 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28dcc6ca-bfc7-4e5a-b201-6bacd53b4b99 service neutron] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:59:22.540465 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28dcc6ca-bfc7-4e5a-b201-6bacd53b4b99 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:59:22.542075 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-28dcc6ca-bfc7-4e5a-b201-6bacd53b4b99 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:59:22.561780 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:59:22.659715 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28dcc6ca-bfc7-4e5a-b201-6bacd53b4b99 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:59:22.661177 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28dcc6ca-bfc7-4e5a-b201-6bacd53b4b99 service neutron] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:59:22.680388 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 678/1328] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 10 11:59:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 11:59:24.937221 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7af82c7-8475-46bf-9f08-26a92b425767 None None] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:59:24.937745 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7af82c7-8475-46bf-9f08-26a92b425767 None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:59:24.937745 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7af82c7-8475-46bf-9f08-26a92b425767 None None] PATH_INFO: `/` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:59:24.939470 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 651/1329] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 10 11:59:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:59:24.944708 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195632ac-5973-4265-8927-5973c0055a55 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:59:24.945127 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195632ac-5973-4265-8927-5973c0055a55 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:59:24.945237 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195632ac-5973-4265-8927-5973c0055a55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:59:25.014018 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-195632ac-5973-4265-8927-5973c0055a55 None None] MFA Rules not processed for user `531d202e4dea4ee38988bfd8b9e26751`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:59:25.015182 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-195632ac-5973-4265-8927-5973c0055a55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:59:25.084043 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:59:25.117536 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 679/1330] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 10 11:59:24 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 11:59:25.401978 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7af82c7-8475-46bf-9f08-26a92b425767 None None] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:59:25.403850 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-f7af82c7-8475-46bf-9f08-26a92b425767 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:59:25.527774 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7af82c7-8475-46bf-9f08-26a92b425767 None None] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:59:25.553055 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:59:25.613438 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f86e5b6-cb0a-4998-90f9-91f178c4238e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58b1c7c85302421cba823ace4201861f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad3772f7b6a47c28b7ba37a9316a3c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:59:25.614304 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f86e5b6-cb0a-4998-90f9-91f178c4238e service neutron] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:59:25.614573 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f86e5b6-cb0a-4998-90f9-91f178c4238e service neutron] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:59:25.614869 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f86e5b6-cb0a-4998-90f9-91f178c4238e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:59:25.616463 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-6f86e5b6-cb0a-4998-90f9-91f178c4238e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:59:25.727946 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f86e5b6-cb0a-4998-90f9-91f178c4238e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:59:25.729391 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f86e5b6-cb0a-4998-90f9-91f178c4238e service neutron] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:59:25.751853 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 652/1331] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 10 11:59:25 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 10 11:59:28.354995 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d5093a-d886-4e6a-9013-944cb2d9d7db None None] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:59:28.355649 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d5093a-d886-4e6a-9013-944cb2d9d7db None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:59:28.355649 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d5093a-d886-4e6a-9013-944cb2d9d7db None None] PATH_INFO: `/` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:59:28.356935 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 680/1332] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 10 11:59:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:59:28.362432 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cfc6e4-08ca-45d3-9efb-40f935fd7f8d None None] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:59:28.362914 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cfc6e4-08ca-45d3-9efb-40f935fd7f8d None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:59:28.363005 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cfc6e4-08ca-45d3-9efb-40f935fd7f8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:59:28.438133 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:59:28.441310 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.auth.core [None req-76cfc6e4-08ca-45d3-9efb-40f935fd7f8d None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:59:28.442559 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-76cfc6e4-08ca-45d3-9efb-40f935fd7f8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:59:28.545637 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 653/1333] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 10 11:59:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 11:59:28.827704 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d5093a-d886-4e6a-9013-944cb2d9d7db None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:59:28.829236 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-e8d5093a-d886-4e6a-9013-944cb2d9d7db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:59:28.931450 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d5093a-d886-4e6a-9013-944cb2d9d7db None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:59:29.007249 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f534248c-22e2-4fda-b04c-70f77cbb2fed service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58b1c7c85302421cba823ace4201861f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad3772f7b6a47c28b7ba37a9316a3c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:59:29.007854 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f534248c-22e2-4fda-b04c-70f77cbb2fed service neutron] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:59:29.008130 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f534248c-22e2-4fda-b04c-70f77cbb2fed service neutron] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:59:29.008398 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f534248c-22e2-4fda-b04c-70f77cbb2fed service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:59:29.009978 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-f534248c-22e2-4fda-b04c-70f77cbb2fed service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:59:29.040927 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:59:29.126990 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f534248c-22e2-4fda-b04c-70f77cbb2fed service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:59:29.128432 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f534248c-22e2-4fda-b04c-70f77cbb2fed service neutron] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:59:29.146571 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 681/1334] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 10 11:59:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 11:59:31.385088 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df39daf4-d439-47c0-9a12-195ceb5b8e93 None None] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:59:31.385637 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df39daf4-d439-47c0-9a12-195ceb5b8e93 None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:59:31.385637 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df39daf4-d439-47c0-9a12-195ceb5b8e93 None None] PATH_INFO: `/` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:59:31.387587 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 654/1335] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 10 11:59:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:59:31.393454 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df7aca0-3fda-49b7-9508-fceeba641e6b None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:59:31.394011 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df7aca0-3fda-49b7-9508-fceeba641e6b None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:59:31.394096 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df7aca0-3fda-49b7-9508-fceeba641e6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:59:31.465951 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-7df7aca0-3fda-49b7-9508-fceeba641e6b None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:59:31.467212 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-7df7aca0-3fda-49b7-9508-fceeba641e6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:59:31.545552 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:59:31.569896 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 682/1336] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 10 11:59:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 11:59:31.855524 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df39daf4-d439-47c0-9a12-195ceb5b8e93 None None] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:59:31.857213 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-df39daf4-d439-47c0-9a12-195ceb5b8e93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:59:31.967637 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df39daf4-d439-47c0-9a12-195ceb5b8e93 None None] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:59:31.998505 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:59:32.045153 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbf9116-e6ae-46c7-bf0d-7ecb67ab0e99 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58b1c7c85302421cba823ace4201861f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad3772f7b6a47c28b7ba37a9316a3c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:59:32.045772 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbf9116-e6ae-46c7-bf0d-7ecb67ab0e99 service neutron] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:59:32.046055 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbf9116-e6ae-46c7-bf0d-7ecb67ab0e99 service neutron] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:59:32.046328 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbf9116-e6ae-46c7-bf0d-7ecb67ab0e99 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:59:32.048065 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-1dbf9116-e6ae-46c7-bf0d-7ecb67ab0e99 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:59:32.163211 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dbf9116-e6ae-46c7-bf0d-7ecb67ab0e99 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:59:32.165415 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dbf9116-e6ae-46c7-bf0d-7ecb67ab0e99 service neutron] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:59:32.200958 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 655/1337] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 10 11:59:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 11:59:35.232500 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdd1940-d643-4e0e-8284-9321d73a5162 None None] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:59:35.233014 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdd1940-d643-4e0e-8284-9321d73a5162 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:59:35.233014 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdd1940-d643-4e0e-8284-9321d73a5162 None None] PATH_INFO: `/` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:59:35.234362 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 683/1338] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 10 11:59:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:59:35.240592 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a921de1-04fd-40f5-8f81-2bef3260312b None None] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:59:35.241091 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a921de1-04fd-40f5-8f81-2bef3260312b None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:59:35.241262 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a921de1-04fd-40f5-8f81-2bef3260312b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:59:35.311749 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.auth.core [None req-3a921de1-04fd-40f5-8f81-2bef3260312b None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:59:35.312989 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-3a921de1-04fd-40f5-8f81-2bef3260312b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:59:35.322255 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:59:35.414005 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 656/1339] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 10 11:59:35 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 10 11:59:35.696689 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfdd1940-d643-4e0e-8284-9321d73a5162 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:59:35.698276 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-bfdd1940-d643-4e0e-8284-9321d73a5162 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:59:35.809266 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfdd1940-d643-4e0e-8284-9321d73a5162 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:59:35.889477 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90e44c5-f422-4d2c-94c4-a68e0302695d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58b1c7c85302421cba823ace4201861f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad3772f7b6a47c28b7ba37a9316a3c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:59:35.890140 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90e44c5-f422-4d2c-94c4-a68e0302695d service neutron] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:59:35.890467 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90e44c5-f422-4d2c-94c4-a68e0302695d service neutron] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:59:35.890817 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90e44c5-f422-4d2c-94c4-a68e0302695d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:59:35.892574 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-b90e44c5-f422-4d2c-94c4-a68e0302695d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:59:35.937243 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:59:36.025400 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90e44c5-f422-4d2c-94c4-a68e0302695d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:59:36.027416 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90e44c5-f422-4d2c-94c4-a68e0302695d service neutron] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:59:36.048270 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 684/1340] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 10 11:59:35 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 11:59:40.650133 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20e6578-5aad-4abd-b67a-2d7e2dde399d None None] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:59:40.650769 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20e6578-5aad-4abd-b67a-2d7e2dde399d None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:59:40.650769 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20e6578-5aad-4abd-b67a-2d7e2dde399d None None] PATH_INFO: `/` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:59:40.652120 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 657/1341] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 10 11:59:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:59:40.657699 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0176bb31-bc55-4bfa-8759-b84a505f9789 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:59:40.658253 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0176bb31-bc55-4bfa-8759-b84a505f9789 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:59:40.658253 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0176bb31-bc55-4bfa-8759-b84a505f9789 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:59:40.741134 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-0176bb31-bc55-4bfa-8759-b84a505f9789 None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:59:40.742385 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-0176bb31-bc55-4bfa-8759-b84a505f9789 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:59:40.835313 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:59:40.846907 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 685/1342] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 10 11:59:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 10 11:59:41.132247 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20e6578-5aad-4abd-b67a-2d7e2dde399d None None] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:59:41.133804 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-e20e6578-5aad-4abd-b67a-2d7e2dde399d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:59:41.243103 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20e6578-5aad-4abd-b67a-2d7e2dde399d None None] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:59:41.286222 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:59:41.324262 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d847680c-f254-4143-bcd6-aec0bdd3b70f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58b1c7c85302421cba823ace4201861f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad3772f7b6a47c28b7ba37a9316a3c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:59:41.324878 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d847680c-f254-4143-bcd6-aec0bdd3b70f service neutron] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:59:41.325160 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d847680c-f254-4143-bcd6-aec0bdd3b70f service neutron] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:59:41.325436 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d847680c-f254-4143-bcd6-aec0bdd3b70f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:59:41.327138 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-d847680c-f254-4143-bcd6-aec0bdd3b70f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:59:41.453809 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d847680c-f254-4143-bcd6-aec0bdd3b70f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:59:41.455213 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d847680c-f254-4143-bcd6-aec0bdd3b70f service neutron] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:59:41.484434 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 658/1343] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 10 11:59:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 11:59:44.673132 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15d7341-ba4c-43ba-b700-b0c85b012003 None None] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:59:44.673659 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15d7341-ba4c-43ba-b700-b0c85b012003 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:59:44.673659 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15d7341-ba4c-43ba-b700-b0c85b012003 None None] PATH_INFO: `/` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:59:44.675055 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 686/1344] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 10 11:59:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:59:44.680388 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2fbebf-4a35-4363-aafc-19795983092b None None] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:59:44.680810 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2fbebf-4a35-4363-aafc-19795983092b None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:59:44.680948 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2fbebf-4a35-4363-aafc-19795983092b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:59:44.747344 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.auth.core [None req-8a2fbebf-4a35-4363-aafc-19795983092b None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:59:44.748504 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-8a2fbebf-4a35-4363-aafc-19795983092b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:59:44.766802 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:59:44.847774 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 659/1345] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 10 11:59:44 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 10 11:59:45.134442 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15d7341-ba4c-43ba-b700-b0c85b012003 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:59:45.136162 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-d15d7341-ba4c-43ba-b700-b0c85b012003 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:59:45.247922 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15d7341-ba4c-43ba-b700-b0c85b012003 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:59:45.323584 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c29e984-3ea1-4b4a-bfcc-eb1c8056c7fb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58b1c7c85302421cba823ace4201861f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad3772f7b6a47c28b7ba37a9316a3c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:59:45.324683 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c29e984-3ea1-4b4a-bfcc-eb1c8056c7fb service neutron] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:59:45.324964 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c29e984-3ea1-4b4a-bfcc-eb1c8056c7fb service neutron] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:59:45.325238 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c29e984-3ea1-4b4a-bfcc-eb1c8056c7fb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:59:45.326860 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-8c29e984-3ea1-4b4a-bfcc-eb1c8056c7fb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:59:45.372006 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:59:45.444095 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c29e984-3ea1-4b4a-bfcc-eb1c8056c7fb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:59:45.445548 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c29e984-3ea1-4b4a-bfcc-eb1c8056c7fb service neutron] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:59:45.467074 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 687/1346] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 10 11:59:45 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 11:59:49.963094 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14175514-8954-4d87-b6b4-7283e0b1553b None None] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:59:49.963560 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14175514-8954-4d87-b6b4-7283e0b1553b None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:59:49.963697 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14175514-8954-4d87-b6b4-7283e0b1553b None None] PATH_INFO: `/` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:59:49.965667 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 660/1347] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 10 11:59:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:59:49.971776 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817d9d30-51d1-4f60-8ed2-f0ef69ff666c None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:59:49.972291 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817d9d30-51d1-4f60-8ed2-f0ef69ff666c None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:59:49.972291 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817d9d30-51d1-4f60-8ed2-f0ef69ff666c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:59:50.044440 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-817d9d30-51d1-4f60-8ed2-f0ef69ff666c None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:59:50.045687 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-817d9d30-51d1-4f60-8ed2-f0ef69ff666c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:59:50.148168 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:59:50.150686 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 688/1348] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 10 11:59:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 11:59:50.432634 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14175514-8954-4d87-b6b4-7283e0b1553b None None] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 11:59:50.434725 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-14175514-8954-4d87-b6b4-7283e0b1553b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:59:50.544589 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14175514-8954-4d87-b6b4-7283e0b1553b None None] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 11:59:50.592319 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:59:50.617524 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4ee5ca-25c9-47d4-a3ea-8662484ce3f1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58b1c7c85302421cba823ace4201861f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad3772f7b6a47c28b7ba37a9316a3c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 11:59:50.618128 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4ee5ca-25c9-47d4-a3ea-8662484ce3f1 service neutron] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:59:50.618399 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4ee5ca-25c9-47d4-a3ea-8662484ce3f1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:59:50.618683 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4ee5ca-25c9-47d4-a3ea-8662484ce3f1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:59:50.620745 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-3e4ee5ca-25c9-47d4-a3ea-8662484ce3f1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:59:50.728503 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4ee5ca-25c9-47d4-a3ea-8662484ce3f1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 11:59:50.730441 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4ee5ca-25c9-47d4-a3ea-8662484ce3f1 service neutron] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 11:59:50.750087 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 661/1349] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 10 11:59:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 11:59:55.853968 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f045389a-7355-4767-bc48-5baf6af2fe2d None None] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:59:55.854974 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f045389a-7355-4767-bc48-5baf6af2fe2d None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:59:55.856458 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f045389a-7355-4767-bc48-5baf6af2fe2d None None] PATH_INFO: `/` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:59:55.858401 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 689/1350] 10.222.0.46 () {54 vars in 921 bytes} [Mon Oct 10 11:59:55 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 11:59:55.864816 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3d0d91-8da9-49d7-95c9-3c221c250c31 None None] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 11:59:55.865695 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3d0d91-8da9-49d7-95c9-3c221c250c31 None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 11:59:55.866120 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3d0d91-8da9-49d7-95c9-3c221c250c31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 11:59:55.964774 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.auth.core [None req-3d3d0d91-8da9-49d7-95c9-3c221c250c31 None None] MFA Rules not processed for user `8e8f9bb1776d47169ef7febb57c954cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 11:59:55.966366 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-3d3d0d91-8da9-49d7-95c9-3c221c250c31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 11:59:55.990730 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 11:59:56.101095 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 662/1351] 10.222.0.46 () {58 vars in 1018 bytes} [Mon Oct 10 11:59:55 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:00:00.688984 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d1bf50-71e5-4458-b8bf-61de8c441979 None None] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:00:00.689544 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d1bf50-71e5-4458-b8bf-61de8c441979 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:00:00.689544 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d1bf50-71e5-4458-b8bf-61de8c441979 None None] PATH_INFO: `/` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:00:00.691094 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 690/1352] 10.222.0.46 () {54 vars in 921 bytes} [Mon Oct 10 12:00:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 12:00:00.696372 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3ba559-16a7-4123-ac4f-b94697d6a297 None None] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:00:00.696861 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3ba559-16a7-4123-ac4f-b94697d6a297 None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:00:00.697121 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3ba559-16a7-4123-ac4f-b94697d6a297 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:00:00.783681 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.auth.core [None req-0e3ba559-16a7-4123-ac4f-b94697d6a297 None None] MFA Rules not processed for user `8e8f9bb1776d47169ef7febb57c954cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:00:00.785050 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-0e3ba559-16a7-4123-ac4f-b94697d6a297 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:00:00.880981 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:00:00.919057 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 663/1353] 10.222.0.46 () {58 vars in 1018 bytes} [Mon Oct 10 12:00:00 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:00:02.498190 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc16ecac-32b0-49b5-9ae3-e10b23970c69 None None] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:00:02.498886 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc16ecac-32b0-49b5-9ae3-e10b23970c69 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:00:02.498886 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc16ecac-32b0-49b5-9ae3-e10b23970c69 None None] PATH_INFO: `/` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:00:02.500141 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 691/1354] 10.222.0.46 () {54 vars in 921 bytes} [Mon Oct 10 12:00:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 12:00:02.507931 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6fdf31-4862-4142-9aa6-b3b057a80f63 None None] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:00:02.508536 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6fdf31-4862-4142-9aa6-b3b057a80f63 None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:00:02.508536 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6fdf31-4862-4142-9aa6-b3b057a80f63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:00:02.600574 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.auth.core [None req-8f6fdf31-4862-4142-9aa6-b3b057a80f63 None None] MFA Rules not processed for user `8e8f9bb1776d47169ef7febb57c954cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:00:02.601823 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-8f6fdf31-4862-4142-9aa6-b3b057a80f63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:00:02.734387 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 664/1355] 10.222.0.46 () {58 vars in 1018 bytes} [Mon Oct 10 12:00:02 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:00:06.285205 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b7d454-fc41-486d-82eb-454c0d4a2174 None None] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:00:06.286038 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b7d454-fc41-486d-82eb-454c0d4a2174 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:00:06.286469 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b7d454-fc41-486d-82eb-454c0d4a2174 None None] PATH_INFO: `/` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:00:06.292677 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 692/1356] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 10 12:00:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 12:00:06.294401 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54925083-f90b-4bba-a461-75a689387328 None None] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:00:06.294903 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54925083-f90b-4bba-a461-75a689387328 None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:00:06.295009 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54925083-f90b-4bba-a461-75a689387328 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:00:06.346234 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:00:06.387970 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.auth.core [None req-54925083-f90b-4bba-a461-75a689387328 None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:00:06.389249 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-54925083-f90b-4bba-a461-75a689387328 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:00:06.513891 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 665/1357] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 10 12:00:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:00:06.816881 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302093ea-57cd-4468-9b89-8807de7febc0 None None] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:00:06.818763 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302093ea-57cd-4468-9b89-8807de7febc0 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:00:06.819816 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302093ea-57cd-4468-9b89-8807de7febc0 None None] PATH_INFO: `/` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:00:06.823691 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 693/1358] 10.222.0.46 () {54 vars in 961 bytes} [Mon Oct 10 12:00:06 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 12:00:06.830209 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cbaf72-128b-4108-a90a-ef9d865638c3 None None] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:00:06.830498 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cbaf72-128b-4108-a90a-ef9d865638c3 None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:00:06.834738 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cbaf72-128b-4108-a90a-ef9d865638c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:00:06.918469 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.auth.core [None req-c5cbaf72-128b-4108-a90a-ef9d865638c3 None None] MFA Rules not processed for user `5f130bbb69e546ce99b4e14de81d7e54`. Rule list: `[]` (Enabled: `True`). {{(pid=81676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:00:06.921643 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-c5cbaf72-128b-4108-a90a-ef9d865638c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:00:06.974300 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:00:07.045182 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 666/1359] 10.222.0.46 () {58 vars in 1058 bytes} [Mon Oct 10 12:00:06 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 10 12:00:07.052836 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302093ea-57cd-4468-9b89-8807de7febc0 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:00:07.054316 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-302093ea-57cd-4468-9b89-8807de7febc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:00:07.213051 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302093ea-57cd-4468-9b89-8807de7febc0 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:00:07.294425 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:00:07.337632 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ada375-68a4-4dad-a4b4-3f2ae3d3606f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f130bbb69e546ce99b4e14de81d7e54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad3772f7b6a47c28b7ba37a9316a3c1', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:00:07.339252 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ada375-68a4-4dad-a4b4-3f2ae3d3606f service nova] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:00:07.339754 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ada375-68a4-4dad-a4b4-3f2ae3d3606f service nova] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:00:07.340265 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ada375-68a4-4dad-a4b4-3f2ae3d3606f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:00:07.342496 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-c7ada375-68a4-4dad-a4b4-3f2ae3d3606f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:00:07.439515 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ada375-68a4-4dad-a4b4-3f2ae3d3606f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:00:07.441491 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ada375-68a4-4dad-a4b4-3f2ae3d3606f service nova] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:00:07.471171 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 694/1360] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 10 12:00:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 419 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:00:09.892151 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84b67ac-70cb-4cf1-893c-aedd3673fe07 None None] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:00:09.892831 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84b67ac-70cb-4cf1-893c-aedd3673fe07 None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:00:09.892831 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84b67ac-70cb-4cf1-893c-aedd3673fe07 None None] PATH_INFO: `/` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:00:09.894095 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 667/1361] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 10 12:00:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 10 12:00:09.900230 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6210e6fe-093a-4b5a-9100-a04e5a6c69c0 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:00:09.900963 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6210e6fe-093a-4b5a-9100-a04e5a6c69c0 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:00:09.900963 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6210e6fe-093a-4b5a-9100-a04e5a6c69c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:00:09.987267 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-6210e6fe-093a-4b5a-9100-a04e5a6c69c0 None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:00:09.989343 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-6210e6fe-093a-4b5a-9100-a04e5a6c69c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:00:10.033087 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:00:10.098150 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 695/1362] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 10 12:00:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:00:10.390298 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84b67ac-70cb-4cf1-893c-aedd3673fe07 None None] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:00:10.392043 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-b84b67ac-70cb-4cf1-893c-aedd3673fe07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:00:10.571453 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84b67ac-70cb-4cf1-893c-aedd3673fe07 None None] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:00:10.584562 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:00:10.717873 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ef1ef7-fb69-4502-8171-f1b6c02fba44 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f130bbb69e546ce99b4e14de81d7e54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad3772f7b6a47c28b7ba37a9316a3c1', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:00:10.717873 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ef1ef7-fb69-4502-8171-f1b6c02fba44 service nova] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:00:10.719121 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ef1ef7-fb69-4502-8171-f1b6c02fba44 service nova] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:00:10.719121 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ef1ef7-fb69-4502-8171-f1b6c02fba44 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:00:10.721055 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-14ef1ef7-fb69-4502-8171-f1b6c02fba44 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:00:10.813173 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ef1ef7-fb69-4502-8171-f1b6c02fba44 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:00:10.814986 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ef1ef7-fb69-4502-8171-f1b6c02fba44 service nova] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:00:10.833627 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 668/1363] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 10 12:00:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 444 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:00:13.462763 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e49aecf-0c38-4df5-ada5-c42a13f4c436 None None] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:00:13.462763 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e49aecf-0c38-4df5-ada5-c42a13f4c436 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:00:13.462763 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e49aecf-0c38-4df5-ada5-c42a13f4c436 None None] PATH_INFO: `/` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:00:13.464090 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 696/1364] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 10 12:00:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 12:00:13.471268 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4799c2-aa24-48a4-bb37-0472ea7cb7f6 None None] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:00:13.471846 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4799c2-aa24-48a4-bb37-0472ea7cb7f6 None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:00:13.471846 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4799c2-aa24-48a4-bb37-0472ea7cb7f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:00:13.492142 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:00:13.557720 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.auth.core [None req-db4799c2-aa24-48a4-bb37-0472ea7cb7f6 None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:00:13.559174 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-db4799c2-aa24-48a4-bb37-0472ea7cb7f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:00:13.684264 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 669/1365] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 10 12:00:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:00:13.980861 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33a9590-702e-42c8-af05-f7fe7936f512 None None] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:00:13.981180 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33a9590-702e-42c8-af05-f7fe7936f512 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:00:13.981433 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33a9590-702e-42c8-af05-f7fe7936f512 None None] PATH_INFO: `/` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:00:13.983601 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 697/1366] 10.222.0.46 () {54 vars in 961 bytes} [Mon Oct 10 12:00:13 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 12:00:13.991165 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6514a2-5b06-4816-b823-5957542e35f6 None None] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:00:13.991550 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6514a2-5b06-4816-b823-5957542e35f6 None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:00:13.992037 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6514a2-5b06-4816-b823-5957542e35f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:00:14.114774 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.auth.core [None req-ef6514a2-5b06-4816-b823-5957542e35f6 None None] MFA Rules not processed for user `5f130bbb69e546ce99b4e14de81d7e54`. Rule list: `[]` (Enabled: `True`). {{(pid=81676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:00:14.116338 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-ef6514a2-5b06-4816-b823-5957542e35f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:00:14.165750 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:00:14.276994 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 670/1367] 10.222.0.46 () {58 vars in 1058 bytes} [Mon Oct 10 12:00:13 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 287 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:00:14.285102 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33a9590-702e-42c8-af05-f7fe7936f512 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:00:14.287702 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-d33a9590-702e-42c8-af05-f7fe7936f512 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:00:14.453924 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33a9590-702e-42c8-af05-f7fe7936f512 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:00:14.480410 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:00:14.595513 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29a9b88-f3c7-4572-9034-423e4455328b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f130bbb69e546ce99b4e14de81d7e54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad3772f7b6a47c28b7ba37a9316a3c1', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:00:14.596043 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29a9b88-f3c7-4572-9034-423e4455328b service nova] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:00:14.596329 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29a9b88-f3c7-4572-9034-423e4455328b service nova] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:00:14.596588 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29a9b88-f3c7-4572-9034-423e4455328b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:00:14.598231 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-a29a9b88-f3c7-4572-9034-423e4455328b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:00:14.632911 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9ab2c0-a2af-4079-9f3d-a2a2f7518192 None None] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:00:14.633487 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9ab2c0-a2af-4079-9f3d-a2a2f7518192 None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:00:14.633785 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9ab2c0-a2af-4079-9f3d-a2a2f7518192 None None] PATH_INFO: `/` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:00:14.635813 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 671/1368] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 10 12:00:14 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 10 12:00:14.641544 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c1e268-bd2f-464f-ad04-9d4a9fe01a97 None None] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:00:14.641827 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c1e268-bd2f-464f-ad04-9d4a9fe01a97 None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:00:14.642109 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c1e268-bd2f-464f-ad04-9d4a9fe01a97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:00:14.709738 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29a9b88-f3c7-4572-9034-423e4455328b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:00:14.711604 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29a9b88-f3c7-4572-9034-423e4455328b service nova] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:00:14.722626 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.auth.core [None req-17c1e268-bd2f-464f-ad04-9d4a9fe01a97 None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:00:14.724910 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-17c1e268-bd2f-464f-ad04-9d4a9fe01a97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:00:14.735648 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 698/1369] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 10 12:00:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 451 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:00:14.878988 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:00:14.887562 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 672/1370] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 10 12:00:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 10 12:00:15.240403 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcefb64-c740-4c22-bc49-0af8a53c068a None None] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:00:15.241021 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcefb64-c740-4c22-bc49-0af8a53c068a None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:00:15.241674 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcefb64-c740-4c22-bc49-0af8a53c068a None None] PATH_INFO: `/` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:00:15.243472 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 699/1371] 10.222.0.46 () {54 vars in 963 bytes} [Mon Oct 10 12:00:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 12:00:15.249485 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709c0a40-9631-4a56-b19d-cb8c2b0b8bde None None] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:00:15.250012 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709c0a40-9631-4a56-b19d-cb8c2b0b8bde None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:00:15.250422 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709c0a40-9631-4a56-b19d-cb8c2b0b8bde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:00:15.357311 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.auth.core [None req-709c0a40-9631-4a56-b19d-cb8c2b0b8bde None None] MFA Rules not processed for user `d3ae093470484b93b23dcfae1657df9e`. Rule list: `[]` (Enabled: `True`). {{(pid=81676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:00:15.358696 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-709c0a40-9631-4a56-b19d-cb8c2b0b8bde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:00:15.459499 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 673/1372] 10.222.0.46 () {58 vars in 1060 bytes} [Mon Oct 10 12:00:15 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 10 12:00:15.467924 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcefb64-c740-4c22-bc49-0af8a53c068a None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:00:15.469457 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-ddcefb64-c740-4c22-bc49-0af8a53c068a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:00:15.504129 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:00:15.594530 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcefb64-c740-4c22-bc49-0af8a53c068a None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:00:15.680042 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40246b4-35d4-477d-880d-5d357a59aa2a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3ae093470484b93b23dcfae1657df9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad3772f7b6a47c28b7ba37a9316a3c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:00:15.681016 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40246b4-35d4-477d-880d-5d357a59aa2a service cinder] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:00:15.681383 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40246b4-35d4-477d-880d-5d357a59aa2a service cinder] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:00:15.681780 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40246b4-35d4-477d-880d-5d357a59aa2a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:00:15.684133 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-b40246b4-35d4-477d-880d-5d357a59aa2a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:00:15.786843 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:00:15.836242 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40246b4-35d4-477d-880d-5d357a59aa2a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:00:15.837695 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40246b4-35d4-477d-880d-5d357a59aa2a service cinder] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:00:15.859532 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 700/1373] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 10 12:00:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 10 12:00:17.219581 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5745aac5-7b9b-408a-8ecb-0ebbcb014cbb None None] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:00:17.220162 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5745aac5-7b9b-408a-8ecb-0ebbcb014cbb None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:00:17.220162 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5745aac5-7b9b-408a-8ecb-0ebbcb014cbb None None] PATH_INFO: `/` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:00:17.221575 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 674/1374] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 10 12:00:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 12:00:17.227890 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500367f3-b46d-41bb-915f-9cf143019d22 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:00:17.228520 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500367f3-b46d-41bb-915f-9cf143019d22 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:00:17.229277 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500367f3-b46d-41bb-915f-9cf143019d22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:00:17.304637 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-500367f3-b46d-41bb-915f-9cf143019d22 None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:00:17.306052 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-500367f3-b46d-41bb-915f-9cf143019d22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:00:17.413907 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 701/1375] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 10 12:00:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:00:17.705043 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5745aac5-7b9b-408a-8ecb-0ebbcb014cbb None None] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:00:17.706802 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-5745aac5-7b9b-408a-8ecb-0ebbcb014cbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:00:17.801619 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:00:17.880640 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5745aac5-7b9b-408a-8ecb-0ebbcb014cbb None None] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:00:18.032201 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500c8122-8693-4fb9-9e79-db23f82c72d8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f130bbb69e546ce99b4e14de81d7e54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad3772f7b6a47c28b7ba37a9316a3c1', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:00:18.032828 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500c8122-8693-4fb9-9e79-db23f82c72d8 service nova] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:00:18.033104 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500c8122-8693-4fb9-9e79-db23f82c72d8 service nova] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:00:18.033378 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500c8122-8693-4fb9-9e79-db23f82c72d8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:00:18.035048 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-500c8122-8693-4fb9-9e79-db23f82c72d8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:00:18.101241 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:00:18.147713 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500c8122-8693-4fb9-9e79-db23f82c72d8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:00:18.149202 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500c8122-8693-4fb9-9e79-db23f82c72d8 service nova] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:00:18.191184 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 675/1376] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 10 12:00:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 472 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:00:20.677254 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91f358e-c3c8-4bc1-a4d5-c3d38ecf1aae None None] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:00:20.677839 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91f358e-c3c8-4bc1-a4d5-c3d38ecf1aae None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:00:20.677998 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91f358e-c3c8-4bc1-a4d5-c3d38ecf1aae None None] PATH_INFO: `/` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:00:20.679677 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 702/1377] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 10 12:00:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 12:00:20.686407 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7379ad70-992f-4eae-a9ff-b4fb7667dc40 None None] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:00:20.687000 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7379ad70-992f-4eae-a9ff-b4fb7667dc40 None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:00:20.687000 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7379ad70-992f-4eae-a9ff-b4fb7667dc40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:00:20.765662 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.auth.core [None req-7379ad70-992f-4eae-a9ff-b4fb7667dc40 None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:00:20.766882 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-7379ad70-992f-4eae-a9ff-b4fb7667dc40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:00:20.912829 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 676/1378] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 10 12:00:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:00:21.205037 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91f358e-c3c8-4bc1-a4d5-c3d38ecf1aae None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:00:21.206687 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-f91f358e-c3c8-4bc1-a4d5-c3d38ecf1aae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:00:21.249510 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:00:21.384165 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91f358e-c3c8-4bc1-a4d5-c3d38ecf1aae None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:00:21.559143 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:00:21.565613 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fb261c-9ba0-4bef-a37b-9637cd539c68 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f130bbb69e546ce99b4e14de81d7e54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad3772f7b6a47c28b7ba37a9316a3c1', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:00:21.566364 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fb261c-9ba0-4bef-a37b-9637cd539c68 service nova] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:00:21.566708 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fb261c-9ba0-4bef-a37b-9637cd539c68 service nova] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:00:21.566987 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fb261c-9ba0-4bef-a37b-9637cd539c68 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:00:21.568684 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-57fb261c-9ba0-4bef-a37b-9637cd539c68 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:00:21.676953 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57fb261c-9ba0-4bef-a37b-9637cd539c68 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:00:21.678646 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57fb261c-9ba0-4bef-a37b-9637cd539c68 service nova] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:00:21.703988 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 703/1379] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 10 12:00:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 500 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 10 12:00:24.190857 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed8f6c2-daba-46e6-b6a3-e1802bd52695 None None] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:00:24.191438 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed8f6c2-daba-46e6-b6a3-e1802bd52695 None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:00:24.191438 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed8f6c2-daba-46e6-b6a3-e1802bd52695 None None] PATH_INFO: `/` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:00:24.192850 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 677/1380] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 10 12:00:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 10 12:00:24.198733 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c174e6-23d7-495a-9094-aea64a56cc47 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:00:24.199491 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c174e6-23d7-495a-9094-aea64a56cc47 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:00:24.199491 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c174e6-23d7-495a-9094-aea64a56cc47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:00:24.276549 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-a4c174e6-23d7-495a-9094-aea64a56cc47 None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:00:24.277695 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-a4c174e6-23d7-495a-9094-aea64a56cc47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:00:24.380505 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:00:24.581631 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 704/1381] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 10 12:00:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 384 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:00:24.893280 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed8f6c2-daba-46e6-b6a3-e1802bd52695 None None] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:00:24.894887 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-4ed8f6c2-daba-46e6-b6a3-e1802bd52695 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:00:24.924117 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:00:25.051711 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed8f6c2-daba-46e6-b6a3-e1802bd52695 None None] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:00:25.167791 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:00:25.187764 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88009007-c110-4709-b284-34c9168441ad service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f130bbb69e546ce99b4e14de81d7e54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad3772f7b6a47c28b7ba37a9316a3c1', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:00:25.188393 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88009007-c110-4709-b284-34c9168441ad service nova] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:00:25.188669 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88009007-c110-4709-b284-34c9168441ad service nova] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:00:25.188952 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88009007-c110-4709-b284-34c9168441ad service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:00:25.190686 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-88009007-c110-4709-b284-34c9168441ad service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:00:25.427060 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88009007-c110-4709-b284-34c9168441ad service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:00:25.428666 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88009007-c110-4709-b284-34c9168441ad service nova] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:00:25.449832 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 678/1382] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 10 12:00:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 557 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 10 12:00:27.828350 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5b3d15-3b89-46ce-8161-05e6716a5f05 None None] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:00:27.828975 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5b3d15-3b89-46ce-8161-05e6716a5f05 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:00:27.828975 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5b3d15-3b89-46ce-8161-05e6716a5f05 None None] PATH_INFO: `/` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:00:27.830156 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 705/1383] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 10 12:00:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 10 12:00:27.836778 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035d420c-26cf-42e6-af05-5f63a2ed2b2e None None] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:00:27.837568 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035d420c-26cf-42e6-af05-5f63a2ed2b2e None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:00:27.837568 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035d420c-26cf-42e6-af05-5f63a2ed2b2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:00:27.938808 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.auth.core [None req-035d420c-26cf-42e6-af05-5f63a2ed2b2e None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:00:27.940032 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-035d420c-26cf-42e6-af05-5f63a2ed2b2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:00:28.008426 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:00:28.060112 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 679/1384] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 10 12:00:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:00:28.349136 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5b3d15-3b89-46ce-8161-05e6716a5f05 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:00:28.350770 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-ea5b3d15-3b89-46ce-8161-05e6716a5f05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:00:28.527827 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5b3d15-3b89-46ce-8161-05e6716a5f05 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:00:28.590216 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:00:28.660332 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8a66ed-fa2a-490a-9952-9758ea43aed7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f130bbb69e546ce99b4e14de81d7e54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad3772f7b6a47c28b7ba37a9316a3c1', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:00:28.660952 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8a66ed-fa2a-490a-9952-9758ea43aed7 service nova] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:00:28.661225 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8a66ed-fa2a-490a-9952-9758ea43aed7 service nova] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:00:28.661537 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8a66ed-fa2a-490a-9952-9758ea43aed7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:00:28.663234 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-5e8a66ed-fa2a-490a-9952-9758ea43aed7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:00:28.772617 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8a66ed-fa2a-490a-9952-9758ea43aed7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:00:28.774585 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8a66ed-fa2a-490a-9952-9758ea43aed7 service nova] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:00:28.797604 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 706/1385] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 10 12:00:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 449 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:00:31.195982 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5fe15e-a52d-4e5a-8ae3-ff60ddd623b3 None None] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:00:31.196593 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5fe15e-a52d-4e5a-8ae3-ff60ddd623b3 None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:00:31.196593 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5fe15e-a52d-4e5a-8ae3-ff60ddd623b3 None None] PATH_INFO: `/` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:00:31.198099 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 680/1386] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 10 12:00:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 12:00:31.205276 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b09e39-42f4-4bc3-85ca-780c98e2bd3b None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:00:31.205823 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b09e39-42f4-4bc3-85ca-780c98e2bd3b None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:00:31.205930 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b09e39-42f4-4bc3-85ca-780c98e2bd3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:00:31.285754 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-72b09e39-42f4-4bc3-85ca-780c98e2bd3b None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:00:31.287001 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-72b09e39-42f4-4bc3-85ca-780c98e2bd3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:00:31.296269 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:00:31.396504 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 707/1387] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 10 12:00:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:00:31.690111 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5fe15e-a52d-4e5a-8ae3-ff60ddd623b3 None None] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:00:31.691730 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-4b5fe15e-a52d-4e5a-8ae3-ff60ddd623b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:00:31.741931 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee60c5a6-920d-4576-b752-c8684d2b8ff8 None None] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:00:31.742281 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee60c5a6-920d-4576-b752-c8684d2b8ff8 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:00:31.742624 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee60c5a6-920d-4576-b752-c8684d2b8ff8 None None] PATH_INFO: `/` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:00:31.744199 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 708/1388] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 10 12:00:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 10 12:00:31.749867 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e97d8d6-ca9e-459e-99d2-132f956ffa85 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:00:31.750117 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e97d8d6-ca9e-459e-99d2-132f956ffa85 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:00:31.750336 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e97d8d6-ca9e-459e-99d2-132f956ffa85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:00:31.855257 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-3e97d8d6-ca9e-459e-99d2-132f956ffa85 None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:00:31.856310 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-3e97d8d6-ca9e-459e-99d2-132f956ffa85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:00:31.872266 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5fe15e-a52d-4e5a-8ae3-ff60ddd623b3 None None] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:00:31.924785 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:00:31.966434 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:00:31.983570 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 709/1389] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 10 12:00:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 10 12:00:32.014826 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6feb939d-e9f5-46ed-a00b-283220afab0c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f130bbb69e546ce99b4e14de81d7e54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad3772f7b6a47c28b7ba37a9316a3c1', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:00:32.015361 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6feb939d-e9f5-46ed-a00b-283220afab0c service nova] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:00:32.015643 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6feb939d-e9f5-46ed-a00b-283220afab0c service nova] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:00:32.015961 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6feb939d-e9f5-46ed-a00b-283220afab0c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:00:32.017670 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-6feb939d-e9f5-46ed-a00b-283220afab0c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:00:32.118280 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6feb939d-e9f5-46ed-a00b-283220afab0c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:00:32.120030 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6feb939d-e9f5-46ed-a00b-283220afab0c service nova] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:00:32.141987 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 681/1390] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 10 12:00:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 453 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:00:32.304285 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80dd7c2-efa3-409e-a806-aa6764e2840c None None] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:00:32.305277 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80dd7c2-efa3-409e-a806-aa6764e2840c None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:00:32.305277 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80dd7c2-efa3-409e-a806-aa6764e2840c None None] PATH_INFO: `/` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:00:32.307394 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 710/1391] 10.222.0.46 () {54 vars in 962 bytes} [Mon Oct 10 12:00:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 10 12:00:32.313405 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b11e47-8f13-4982-983a-3696f576074f None None] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:00:32.314025 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b11e47-8f13-4982-983a-3696f576074f None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:00:32.314203 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b11e47-8f13-4982-983a-3696f576074f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:00:32.386026 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:00:32.411794 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.auth.core [None req-e1b11e47-8f13-4982-983a-3696f576074f None None] MFA Rules not processed for user `124f4aa8ab244ca2aadf4f2614df51e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:00:32.414120 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-e1b11e47-8f13-4982-983a-3696f576074f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:00:32.543139 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 682/1392] 10.222.0.46 () {58 vars in 1059 bytes} [Mon Oct 10 12:00:32 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 10 12:00:32.553513 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80dd7c2-efa3-409e-a806-aa6764e2840c None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:00:32.555218 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-a80dd7c2-efa3-409e-a806-aa6764e2840c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:00:32.674499 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80dd7c2-efa3-409e-a806-aa6764e2840c None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:00:32.758615 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0870ccf1-ad80-46d8-87c6-3c88fb519cc8 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '124f4aa8ab244ca2aadf4f2614df51e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad3772f7b6a47c28b7ba37a9316a3c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:00:32.759366 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0870ccf1-ad80-46d8-87c6-3c88fb519cc8 service glance] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:00:32.759727 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0870ccf1-ad80-46d8-87c6-3c88fb519cc8 service glance] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:00:32.760096 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0870ccf1-ad80-46d8-87c6-3c88fb519cc8 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:00:32.762021 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-0870ccf1-ad80-46d8-87c6-3c88fb519cc8 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:00:32.815117 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:00:32.893926 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0870ccf1-ad80-46d8-87c6-3c88fb519cc8 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:00:32.895454 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0870ccf1-ad80-46d8-87c6-3c88fb519cc8 service glance] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:00:32.917173 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 711/1393] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 10 12:00:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:00:34.012081 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c64f1c2-833a-4882-920d-0374c4f98aab None None] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:00:34.012923 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c64f1c2-833a-4882-920d-0374c4f98aab None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:00:34.012923 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c64f1c2-833a-4882-920d-0374c4f98aab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:00:34.095445 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.auth.core [None req-5c64f1c2-833a-4882-920d-0374c4f98aab None None] MFA Rules not processed for user `d6430593eecf45d8b891c3b55de4a944`. Rule list: `[]` (Enabled: `True`). {{(pid=81676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:00:34.097380 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-5c64f1c2-833a-4882-920d-0374c4f98aab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:00:34.231041 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 683/1394] 10.222.0.46 () {58 vars in 1009 bytes} [Mon Oct 10 12:00:34 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 10 12:00:34.287963 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e9f14e-6381-42ca-89c1-81010d1c6628 None None] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:00:34.288651 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e9f14e-6381-42ca-89c1-81010d1c6628 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:00:34.288651 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e9f14e-6381-42ca-89c1-81010d1c6628 None None] PATH_INFO: `/` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:00:34.290112 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 712/1395] 10.222.0.46 () {54 vars in 943 bytes} [Mon Oct 10 12:00:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 12:00:34.298429 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0d8ae6-4ea0-4ac9-9c3a-3aab609f3405 None None] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:00:34.298883 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0d8ae6-4ea0-4ac9-9c3a-3aab609f3405 None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:00:34.299265 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0d8ae6-4ea0-4ac9-9c3a-3aab609f3405 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:00:34.310382 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:00:34.396095 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.auth.core [None req-0e0d8ae6-4ea0-4ac9-9c3a-3aab609f3405 None None] MFA Rules not processed for user `15ef3d53ea484237adb0b3d64fdc5480`. Rule list: `[]` (Enabled: `True`). {{(pid=81676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:00:34.397581 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-0e0d8ae6-4ea0-4ac9-9c3a-3aab609f3405 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:00:34.564223 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 684/1396] 10.222.0.46 () {58 vars in 1040 bytes} [Mon Oct 10 12:00:34 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 267 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:00:34.576771 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e9f14e-6381-42ca-89c1-81010d1c6628 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:00:34.579644 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-d9e9f14e-6381-42ca-89c1-81010d1c6628 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:00:34.784997 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e9f14e-6381-42ca-89c1-81010d1c6628 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:00:34.803827 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:00:34.914760 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b9a058-c393-41dd-90be-62685634111d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15ef3d53ea484237adb0b3d64fdc5480', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad3772f7b6a47c28b7ba37a9316a3c1', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:00:34.915376 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b9a058-c393-41dd-90be-62685634111d service swift] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:00:34.915860 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b9a058-c393-41dd-90be-62685634111d service swift] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:00:34.916128 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b9a058-c393-41dd-90be-62685634111d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:00:34.917943 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-35b9a058-c393-41dd-90be-62685634111d service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:00:34.921563 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f7fc38-3135-456c-bc68-b5551eda409d None None] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:00:34.921843 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f7fc38-3135-456c-bc68-b5551eda409d None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:00:34.922113 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f7fc38-3135-456c-bc68-b5551eda409d None None] PATH_INFO: `/` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:00:34.923495 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 685/1397] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 10 12:00:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 10 12:00:34.931423 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa28549c-b4c2-46f9-897a-f464e6c07454 None None] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:00:34.931967 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa28549c-b4c2-46f9-897a-f464e6c07454 None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:00:34.932453 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa28549c-b4c2-46f9-897a-f464e6c07454 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:00:35.018406 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.auth.core [None req-fa28549c-b4c2-46f9-897a-f464e6c07454 None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:00:35.020747 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-fa28549c-b4c2-46f9-897a-f464e6c07454 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:00:35.032979 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b9a058-c393-41dd-90be-62685634111d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:00:35.067827 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b9a058-c393-41dd-90be-62685634111d service swift] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:00:35.068970 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:00:35.069166 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 713/1398] 10.222.0.46 () {60 vars in 1384 bytes} [Mon Oct 10 12:00:34 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 472 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 10 12:00:35.157430 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 686/1399] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 10 12:00:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 10 12:00:35.513997 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b9a058-c393-41dd-90be-62685634111d service swift] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:00:35.515732 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-35b9a058-c393-41dd-90be-62685634111d service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:00:35.571175 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:00:35.627554 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b9a058-c393-41dd-90be-62685634111d service swift] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:00:35.794358 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b7cc34-2d91-41ee-907b-f122bcd90ca2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f130bbb69e546ce99b4e14de81d7e54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad3772f7b6a47c28b7ba37a9316a3c1', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:00:35.794358 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b7cc34-2d91-41ee-907b-f122bcd90ca2 service nova] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:00:35.794358 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b7cc34-2d91-41ee-907b-f122bcd90ca2 service nova] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:00:35.794358 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b7cc34-2d91-41ee-907b-f122bcd90ca2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:00:35.794940 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-f7b7cc34-2d91-41ee-907b-f122bcd90ca2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:00:35.918271 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b7cc34-2d91-41ee-907b-f122bcd90ca2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:00:35.920024 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b7cc34-2d91-41ee-907b-f122bcd90ca2 service nova] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:00:35.943823 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 714/1400] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 10 12:00:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 430 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 10 12:00:36.173270 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:00:39.269164 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce8c423-7a4e-4a0b-bd31-cc6579cefadf None None] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:00:39.269958 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce8c423-7a4e-4a0b-bd31-cc6579cefadf None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:00:39.269958 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce8c423-7a4e-4a0b-bd31-cc6579cefadf None None] PATH_INFO: `/` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:00:39.271404 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 687/1401] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 10 12:00:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 12:00:39.277717 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741dffab-6046-48f8-b5ad-9d9e8fdfdacf None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:00:39.278228 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741dffab-6046-48f8-b5ad-9d9e8fdfdacf None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:00:39.278314 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741dffab-6046-48f8-b5ad-9d9e8fdfdacf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:00:39.366683 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-741dffab-6046-48f8-b5ad-9d9e8fdfdacf None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:00:39.371296 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-741dffab-6046-48f8-b5ad-9d9e8fdfdacf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:00:39.535006 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 715/1402] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 10 12:00:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 258 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:00:39.820920 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce8c423-7a4e-4a0b-bd31-cc6579cefadf None None] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:00:39.823891 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-8ce8c423-7a4e-4a0b-bd31-cc6579cefadf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:00:40.148419 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:00:40.151930 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce8c423-7a4e-4a0b-bd31-cc6579cefadf None None] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:00:40.395384 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31214577-93fe-45c3-921e-44306fd846db service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f130bbb69e546ce99b4e14de81d7e54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad3772f7b6a47c28b7ba37a9316a3c1', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:00:40.396100 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31214577-93fe-45c3-921e-44306fd846db service nova] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:00:40.396352 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31214577-93fe-45c3-921e-44306fd846db service nova] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:00:40.396659 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31214577-93fe-45c3-921e-44306fd846db service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:00:40.398915 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-31214577-93fe-45c3-921e-44306fd846db service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:00:40.562746 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31214577-93fe-45c3-921e-44306fd846db service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:00:40.568791 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31214577-93fe-45c3-921e-44306fd846db service nova] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:00:40.589945 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 688/1403] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 10 12:00:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 770 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:00:43.104409 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05437ab6-f167-449a-a715-234e4d165ec8 None None] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:00:43.105465 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05437ab6-f167-449a-a715-234e4d165ec8 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:00:43.105991 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05437ab6-f167-449a-a715-234e4d165ec8 None None] PATH_INFO: `/` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:00:43.107684 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 716/1404] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 10 12:00:43 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 10 12:00:43.113773 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04342670-4fcd-4825-9e2e-e95339ca7f9b None None] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:00:43.114238 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04342670-4fcd-4825-9e2e-e95339ca7f9b None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:00:43.114331 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04342670-4fcd-4825-9e2e-e95339ca7f9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:00:43.119990 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:00:43.205521 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.auth.core [None req-04342670-4fcd-4825-9e2e-e95339ca7f9b None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:00:43.206793 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-04342670-4fcd-4825-9e2e-e95339ca7f9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:00:43.350862 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 689/1405] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 10 12:00:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 10 12:00:43.623239 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05437ab6-f167-449a-a715-234e4d165ec8 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:00:43.625113 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-05437ab6-f167-449a-a715-234e4d165ec8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:00:43.675917 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:00:43.803650 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05437ab6-f167-449a-a715-234e4d165ec8 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:00:43.977540 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0129e711-142d-4552-8212-1f738fa00f4f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f130bbb69e546ce99b4e14de81d7e54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad3772f7b6a47c28b7ba37a9316a3c1', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:00:43.978274 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0129e711-142d-4552-8212-1f738fa00f4f service nova] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:00:43.978560 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0129e711-142d-4552-8212-1f738fa00f4f service nova] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:00:43.978880 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0129e711-142d-4552-8212-1f738fa00f4f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:00:43.980600 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-0129e711-142d-4552-8212-1f738fa00f4f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:00:43.983582 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:00:44.087859 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0129e711-142d-4552-8212-1f738fa00f4f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:00:44.089893 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0129e711-142d-4552-8212-1f738fa00f4f service nova] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:00:44.122885 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 717/1406] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 10 12:00:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 500 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:00:46.987660 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ad167d-9965-466e-9106-389195c5d7c8 None None] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:00:46.988264 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ad167d-9965-466e-9106-389195c5d7c8 None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:00:46.988264 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ad167d-9965-466e-9106-389195c5d7c8 None None] PATH_INFO: `/` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:00:46.989988 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 690/1407] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 10 12:00:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 12:00:46.995451 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d27ae37-9a41-4ad1-8464-6023b3b21127 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:00:46.995981 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d27ae37-9a41-4ad1-8464-6023b3b21127 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:00:46.995981 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d27ae37-9a41-4ad1-8464-6023b3b21127 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:00:47.076051 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-5d27ae37-9a41-4ad1-8464-6023b3b21127 None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:00:47.077582 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-5d27ae37-9a41-4ad1-8464-6023b3b21127 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:00:47.191872 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:00:47.254779 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 718/1408] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 10 12:00:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 10 12:00:47.540697 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ad167d-9965-466e-9106-389195c5d7c8 None None] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:00:47.542375 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-a1ad167d-9965-466e-9106-389195c5d7c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:00:47.640559 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:00:47.725079 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ad167d-9965-466e-9106-389195c5d7c8 None None] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:00:47.854996 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998c5021-39be-4e39-b572-4b22f89dae5a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f130bbb69e546ce99b4e14de81d7e54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad3772f7b6a47c28b7ba37a9316a3c1', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:00:47.855868 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998c5021-39be-4e39-b572-4b22f89dae5a service nova] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:00:47.856176 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998c5021-39be-4e39-b572-4b22f89dae5a service nova] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:00:47.856490 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998c5021-39be-4e39-b572-4b22f89dae5a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:00:47.858195 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-998c5021-39be-4e39-b572-4b22f89dae5a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:00:47.919483 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:00:47.972378 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998c5021-39be-4e39-b572-4b22f89dae5a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:00:47.974432 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998c5021-39be-4e39-b572-4b22f89dae5a service nova] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:00:47.996775 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 691/1409] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 10 12:00:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 457 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:00:57.239368 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=9 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:00:57.993985 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=25 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:01:01.134231 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f0ebb3-4a43-4024-8bf0-2c05c80012f9 None None] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:01:01.134901 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f0ebb3-4a43-4024-8bf0-2c05c80012f9 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:01:01.134901 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f0ebb3-4a43-4024-8bf0-2c05c80012f9 None None] PATH_INFO: `/` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:01:01.136168 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 719/1410] 10.222.0.46 () {54 vars in 918 bytes} [Mon Oct 10 12:01:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 10 12:01:01.141852 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddea288e-7089-4ca3-9990-2ed19424caa0 None None] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:01:01.142390 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddea288e-7089-4ca3-9990-2ed19424caa0 None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:01:01.142390 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddea288e-7089-4ca3-9990-2ed19424caa0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:01:01.222597 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.auth.core [None req-ddea288e-7089-4ca3-9990-2ed19424caa0 None None] MFA Rules not processed for user `8e8f9bb1776d47169ef7febb57c954cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:01:01.223867 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-ddea288e-7089-4ca3-9990-2ed19424caa0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:01:01.339568 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 692/1411] 10.222.0.46 () {58 vars in 1015 bytes} [Mon Oct 10 12:01:01 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 10 12:01:01.355273 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c437ecae-45e7-4732-b2c2-39fe7cb0fbac None None] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:01:01.355607 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c437ecae-45e7-4732-b2c2-39fe7cb0fbac None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:01:01.355987 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c437ecae-45e7-4732-b2c2-39fe7cb0fbac None None] PATH_INFO: `/` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:01:01.357420 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 720/1412] 10.222.0.46 () {54 vars in 918 bytes} [Mon Oct 10 12:01:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 12:01:01.362843 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cf0a76-00d6-4f05-8b0d-77364728dfae None None] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:01:01.363155 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cf0a76-00d6-4f05-8b0d-77364728dfae None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:01:01.363432 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cf0a76-00d6-4f05-8b0d-77364728dfae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:01:01.444086 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:01:01.447027 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.auth.core [None req-e4cf0a76-00d6-4f05-8b0d-77364728dfae None None] MFA Rules not processed for user `5f130bbb69e546ce99b4e14de81d7e54`. Rule list: `[]` (Enabled: `True`). {{(pid=81676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:01:01.448273 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-e4cf0a76-00d6-4f05-8b0d-77364728dfae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:01:01.592012 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 693/1413] 10.222.0.46 () {58 vars in 1015 bytes} [Mon Oct 10 12:01:01 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 10 12:01:04.460804 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c45a1c-afd5-47cc-a8ed-729a9308dfa0 None None] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:01:04.461491 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c45a1c-afd5-47cc-a8ed-729a9308dfa0 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:01:04.461491 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c45a1c-afd5-47cc-a8ed-729a9308dfa0 None None] PATH_INFO: `/` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:01:04.463186 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 721/1414] 127.0.0.1 () {54 vars in 917 bytes} [Mon Oct 10 12:01:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 12:01:04.472894 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d296879d-83fb-456c-aba3-025b77247741 None None] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:01:04.472894 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d296879d-83fb-456c-aba3-025b77247741 None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:01:04.473623 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d296879d-83fb-456c-aba3-025b77247741 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:01:04.568485 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.auth.core [None req-d296879d-83fb-456c-aba3-025b77247741 None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:01:04.569703 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-d296879d-83fb-456c-aba3-025b77247741 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:01:04.627983 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:01:04.672455 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 694/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Oct 10 12:01:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:01:04.964506 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c45a1c-afd5-47cc-a8ed-729a9308dfa0 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:01:04.966019 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-07c45a1c-afd5-47cc-a8ed-729a9308dfa0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:01:05.123851 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c45a1c-afd5-47cc-a8ed-729a9308dfa0 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:01:05.214095 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:01:05.256869 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7814f875-af4a-4af6-a22b-63081570a9ce service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f130bbb69e546ce99b4e14de81d7e54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad3772f7b6a47c28b7ba37a9316a3c1', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:01:05.257606 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7814f875-af4a-4af6-a22b-63081570a9ce service nova] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:01:05.258332 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7814f875-af4a-4af6-a22b-63081570a9ce service nova] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:01:05.259186 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7814f875-af4a-4af6-a22b-63081570a9ce service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:01:05.261941 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-7814f875-af4a-4af6-a22b-63081570a9ce service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:01:05.370741 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7814f875-af4a-4af6-a22b-63081570a9ce service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:01:05.372351 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7814f875-af4a-4af6-a22b-63081570a9ce service nova] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:01:05.393378 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 722/1416] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 10 12:01:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 429 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 10 12:01:06.564032 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efb5c5a-d781-436f-af03-392b2df37128 None None] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:01:06.564790 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efb5c5a-d781-436f-af03-392b2df37128 None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:01:06.564790 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efb5c5a-d781-436f-af03-392b2df37128 None None] PATH_INFO: `/` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:01:06.566228 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 695/1417] 127.0.0.1 () {54 vars in 901 bytes} [Mon Oct 10 12:01:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 12:01:06.573708 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a651d45-9069-4f40-93d1-badc0780f125 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:01:06.574375 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a651d45-9069-4f40-93d1-badc0780f125 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:01:06.574375 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a651d45-9069-4f40-93d1-badc0780f125 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:01:06.653833 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-6a651d45-9069-4f40-93d1-badc0780f125 None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:01:06.655085 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-6a651d45-9069-4f40-93d1-badc0780f125 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:01:06.695972 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:01:06.790492 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 723/1418] 127.0.0.1 () {58 vars in 998 bytes} [Mon Oct 10 12:01:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:01:06.970470 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efb5c5a-d781-436f-af03-392b2df37128 None None] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:01:06.971997 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-7efb5c5a-d781-436f-af03-392b2df37128 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:01:07.134481 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efb5c5a-d781-436f-af03-392b2df37128 None None] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:01:07.187385 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:01:07.272534 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86565c69-4d4f-41fc-bbb8-90dbd707fac7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f130bbb69e546ce99b4e14de81d7e54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad3772f7b6a47c28b7ba37a9316a3c1', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:01:07.273140 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86565c69-4d4f-41fc-bbb8-90dbd707fac7 service nova] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:01:07.273416 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86565c69-4d4f-41fc-bbb8-90dbd707fac7 service nova] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:01:07.273685 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86565c69-4d4f-41fc-bbb8-90dbd707fac7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:01:07.275372 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-86565c69-4d4f-41fc-bbb8-90dbd707fac7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:01:07.368799 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86565c69-4d4f-41fc-bbb8-90dbd707fac7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:01:07.370428 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86565c69-4d4f-41fc-bbb8-90dbd707fac7 service nova] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:01:07.389373 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 696/1419] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 10 12:01:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 419 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 10 12:01:08.562698 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f166cdb-1bf9-482d-b0ed-90000ebd7513 None None] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:01:08.563343 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f166cdb-1bf9-482d-b0ed-90000ebd7513 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:01:08.563343 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f166cdb-1bf9-482d-b0ed-90000ebd7513 None None] PATH_INFO: `/` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:01:08.564612 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 724/1420] 127.0.0.1 () {54 vars in 901 bytes} [Mon Oct 10 12:01:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 12:01:08.571686 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca5508d-2645-4eaf-a5e6-fe8b84617672 None None] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:01:08.572292 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca5508d-2645-4eaf-a5e6-fe8b84617672 None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:01:08.572292 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca5508d-2645-4eaf-a5e6-fe8b84617672 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:01:08.635002 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:01:08.663555 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.auth.core [None req-fca5508d-2645-4eaf-a5e6-fe8b84617672 None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:01:08.665396 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-fca5508d-2645-4eaf-a5e6-fe8b84617672 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:01:08.769836 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 697/1421] 127.0.0.1 () {58 vars in 998 bytes} [Mon Oct 10 12:01:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:01:08.924975 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f166cdb-1bf9-482d-b0ed-90000ebd7513 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:01:08.926486 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-4f166cdb-1bf9-482d-b0ed-90000ebd7513 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:01:09.078376 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f166cdb-1bf9-482d-b0ed-90000ebd7513 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:01:09.091686 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:01:09.195759 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e1a235-4a99-48bd-af11-5ca39bb3cf6a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f130bbb69e546ce99b4e14de81d7e54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad3772f7b6a47c28b7ba37a9316a3c1', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:01:09.196373 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e1a235-4a99-48bd-af11-5ca39bb3cf6a service nova] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:01:09.196649 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e1a235-4a99-48bd-af11-5ca39bb3cf6a service nova] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:01:09.196917 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e1a235-4a99-48bd-af11-5ca39bb3cf6a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:01:09.198506 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-76e1a235-4a99-48bd-af11-5ca39bb3cf6a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:01:09.293375 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e1a235-4a99-48bd-af11-5ca39bb3cf6a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:01:09.295034 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e1a235-4a99-48bd-af11-5ca39bb3cf6a service nova] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:01:09.313164 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 725/1422] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 10 12:01:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:01:10.438922 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349539e9-416c-404f-81dd-474e1d9de934 None None] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:01:10.439539 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349539e9-416c-404f-81dd-474e1d9de934 None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:01:10.439785 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349539e9-416c-404f-81dd-474e1d9de934 None None] PATH_INFO: `/` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:01:10.441613 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 698/1423] 127.0.0.1 () {54 vars in 901 bytes} [Mon Oct 10 12:01:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 12:01:10.449625 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f06786-c79b-4e16-863c-08f3ef0db007 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:01:10.450243 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f06786-c79b-4e16-863c-08f3ef0db007 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:01:10.450243 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f06786-c79b-4e16-863c-08f3ef0db007 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:01:10.473511 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:01:10.537411 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-71f06786-c79b-4e16-863c-08f3ef0db007 None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:01:10.538635 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-71f06786-c79b-4e16-863c-08f3ef0db007 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:01:10.644060 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 726/1424] 127.0.0.1 () {58 vars in 998 bytes} [Mon Oct 10 12:01:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:01:10.796981 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349539e9-416c-404f-81dd-474e1d9de934 None None] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:01:10.798417 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-349539e9-416c-404f-81dd-474e1d9de934 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:01:10.919020 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:01:10.960892 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349539e9-416c-404f-81dd-474e1d9de934 None None] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:01:11.088404 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d6c8b4-eb42-4565-ada1-9cbb4437e95c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f130bbb69e546ce99b4e14de81d7e54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad3772f7b6a47c28b7ba37a9316a3c1', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:01:11.088929 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d6c8b4-eb42-4565-ada1-9cbb4437e95c service nova] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:01:11.089208 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d6c8b4-eb42-4565-ada1-9cbb4437e95c service nova] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:01:11.089479 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d6c8b4-eb42-4565-ada1-9cbb4437e95c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:01:11.091149 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-e6d6c8b4-eb42-4565-ada1-9cbb4437e95c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:01:11.172056 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:01:11.187153 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d6c8b4-eb42-4565-ada1-9cbb4437e95c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:01:11.188760 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d6c8b4-eb42-4565-ada1-9cbb4437e95c service nova] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:01:11.207825 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 699/1425] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 10 12:01:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:01:12.362885 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c405aec-3a3c-4e24-b7a0-c84aceb39a84 None None] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:01:12.363463 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c405aec-3a3c-4e24-b7a0-c84aceb39a84 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:01:12.363463 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c405aec-3a3c-4e24-b7a0-c84aceb39a84 None None] PATH_INFO: `/` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:01:12.364784 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 727/1426] 127.0.0.1 () {54 vars in 901 bytes} [Mon Oct 10 12:01:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 12:01:12.372222 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f39979-2004-4e40-abde-ccee5c80961d None None] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:01:12.372747 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f39979-2004-4e40-abde-ccee5c80961d None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:01:12.372747 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f39979-2004-4e40-abde-ccee5c80961d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:01:12.450139 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.auth.core [None req-75f39979-2004-4e40-abde-ccee5c80961d None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:01:12.451488 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-75f39979-2004-4e40-abde-ccee5c80961d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:01:12.557772 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 700/1427] 127.0.0.1 () {58 vars in 998 bytes} [Mon Oct 10 12:01:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:01:12.709958 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c405aec-3a3c-4e24-b7a0-c84aceb39a84 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:01:12.711512 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-4c405aec-3a3c-4e24-b7a0-c84aceb39a84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:01:12.801851 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:01:12.868320 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c405aec-3a3c-4e24-b7a0-c84aceb39a84 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:01:12.993695 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74dd2e5f-d781-44fa-99eb-e01ff48624ef service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f130bbb69e546ce99b4e14de81d7e54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad3772f7b6a47c28b7ba37a9316a3c1', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:01:12.994310 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dd2e5f-d781-44fa-99eb-e01ff48624ef service nova] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:01:12.994588 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dd2e5f-d781-44fa-99eb-e01ff48624ef service nova] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:01:12.994893 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dd2e5f-d781-44fa-99eb-e01ff48624ef service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:01:12.996524 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-74dd2e5f-d781-44fa-99eb-e01ff48624ef service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:01:13.054419 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:01:13.101437 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74dd2e5f-d781-44fa-99eb-e01ff48624ef service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:01:13.103721 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74dd2e5f-d781-44fa-99eb-e01ff48624ef service nova] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:01:13.123568 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 728/1428] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 10 12:01:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:01:14.263126 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12547f4-1d4f-41db-ac65-56637c958c16 None None] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:01:14.263797 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12547f4-1d4f-41db-ac65-56637c958c16 None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:01:14.263891 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12547f4-1d4f-41db-ac65-56637c958c16 None None] PATH_INFO: `/` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:01:14.265454 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 701/1429] 127.0.0.1 () {54 vars in 901 bytes} [Mon Oct 10 12:01:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 12:01:14.272537 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef508e0-e1f8-4230-bc5e-abcb2461f53e None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:01:14.273041 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef508e0-e1f8-4230-bc5e-abcb2461f53e None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:01:14.273286 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef508e0-e1f8-4230-bc5e-abcb2461f53e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:01:14.376603 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-fef508e0-e1f8-4230-bc5e-abcb2461f53e None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:01:14.378395 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-fef508e0-e1f8-4230-bc5e-abcb2461f53e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:01:14.495053 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 729/1430] 127.0.0.1 () {58 vars in 998 bytes} [Mon Oct 10 12:01:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:01:14.647492 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12547f4-1d4f-41db-ac65-56637c958c16 None None] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:01:14.649860 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-e12547f4-1d4f-41db-ac65-56637c958c16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:01:14.712751 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:01:14.817004 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12547f4-1d4f-41db-ac65-56637c958c16 None None] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:01:14.953486 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb92924f-0c15-46b1-a435-d158b91d19dc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f130bbb69e546ce99b4e14de81d7e54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad3772f7b6a47c28b7ba37a9316a3c1', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:01:14.954090 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb92924f-0c15-46b1-a435-d158b91d19dc service nova] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:01:14.954743 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb92924f-0c15-46b1-a435-d158b91d19dc service nova] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:01:14.954743 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb92924f-0c15-46b1-a435-d158b91d19dc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:01:14.956269 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-eb92924f-0c15-46b1-a435-d158b91d19dc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:01:14.964632 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:01:15.050904 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb92924f-0c15-46b1-a435-d158b91d19dc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:01:15.053115 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb92924f-0c15-46b1-a435-d158b91d19dc service nova] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:01:15.084723 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 702/1431] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 10 12:01:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 438 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:01:16.276646 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a769a24f-3685-440d-8cb7-4720f9a6c913 None None] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:01:16.277249 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a769a24f-3685-440d-8cb7-4720f9a6c913 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:01:16.277249 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a769a24f-3685-440d-8cb7-4720f9a6c913 None None] PATH_INFO: `/` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:01:16.278550 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 730/1432] 127.0.0.1 () {54 vars in 901 bytes} [Mon Oct 10 12:01:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 12:01:16.286825 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580e7c2e-f976-4f7d-b407-69238e536473 None None] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:01:16.287614 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580e7c2e-f976-4f7d-b407-69238e536473 None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:01:16.287948 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580e7c2e-f976-4f7d-b407-69238e536473 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:01:16.377503 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.auth.core [None req-580e7c2e-f976-4f7d-b407-69238e536473 None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:01:16.378767 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-580e7c2e-f976-4f7d-b407-69238e536473 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:01:16.452597 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:01:16.487626 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 703/1433] 127.0.0.1 () {58 vars in 998 bytes} [Mon Oct 10 12:01:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:01:16.695063 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a769a24f-3685-440d-8cb7-4720f9a6c913 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:01:16.697255 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-a769a24f-3685-440d-8cb7-4720f9a6c913 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:01:16.715436 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:01:16.863604 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a769a24f-3685-440d-8cb7-4720f9a6c913 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:01:16.957756 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:01:16.985092 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7cb570-1ba4-4e54-aadb-0fd63dbe3125 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f130bbb69e546ce99b4e14de81d7e54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad3772f7b6a47c28b7ba37a9316a3c1', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:01:16.985704 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7cb570-1ba4-4e54-aadb-0fd63dbe3125 service nova] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:01:16.985988 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7cb570-1ba4-4e54-aadb-0fd63dbe3125 service nova] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:01:16.986266 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7cb570-1ba4-4e54-aadb-0fd63dbe3125 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:01:16.988201 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-be7cb570-1ba4-4e54-aadb-0fd63dbe3125 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:01:17.085012 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7cb570-1ba4-4e54-aadb-0fd63dbe3125 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:01:17.086650 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7cb570-1ba4-4e54-aadb-0fd63dbe3125 service nova] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:01:17.106171 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 731/1434] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 10 12:01:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:01:18.249633 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df45a826-39a2-4e10-8c16-38cec9e37766 None None] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:01:18.250225 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df45a826-39a2-4e10-8c16-38cec9e37766 None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:01:18.250225 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df45a826-39a2-4e10-8c16-38cec9e37766 None None] PATH_INFO: `/` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:01:18.251709 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 704/1435] 127.0.0.1 () {54 vars in 901 bytes} [Mon Oct 10 12:01:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 10 12:01:18.259275 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a9019d-bb8f-4fb4-8b80-c8f81ea8a01b None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:01:18.259898 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a9019d-bb8f-4fb4-8b80-c8f81ea8a01b None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:01:18.259898 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a9019d-bb8f-4fb4-8b80-c8f81ea8a01b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:01:18.345574 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-c7a9019d-bb8f-4fb4-8b80-c8f81ea8a01b None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:01:18.346826 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-c7a9019d-bb8f-4fb4-8b80-c8f81ea8a01b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:01:18.399223 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:01:18.467729 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 732/1436] 127.0.0.1 () {58 vars in 998 bytes} [Mon Oct 10 12:01:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:01:18.626371 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df45a826-39a2-4e10-8c16-38cec9e37766 None None] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:01:18.628126 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-df45a826-39a2-4e10-8c16-38cec9e37766 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:01:18.801417 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df45a826-39a2-4e10-8c16-38cec9e37766 None None] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:01:18.859175 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:01:18.923144 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a631c93d-56a6-4f47-b784-e2c05597fa48 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f130bbb69e546ce99b4e14de81d7e54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad3772f7b6a47c28b7ba37a9316a3c1', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:01:18.923749 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a631c93d-56a6-4f47-b784-e2c05597fa48 service nova] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:01:18.924020 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a631c93d-56a6-4f47-b784-e2c05597fa48 service nova] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:01:18.924285 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a631c93d-56a6-4f47-b784-e2c05597fa48 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:01:18.925911 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-a631c93d-56a6-4f47-b784-e2c05597fa48 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:01:19.031029 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a631c93d-56a6-4f47-b784-e2c05597fa48 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:01:19.033884 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a631c93d-56a6-4f47-b784-e2c05597fa48 service nova] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:01:19.057229 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 705/1437] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 10 12:01:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 431 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:01:20.204621 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7c8abe-4435-4027-89b1-f70959308025 None None] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:01:20.205255 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7c8abe-4435-4027-89b1-f70959308025 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:01:20.205255 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7c8abe-4435-4027-89b1-f70959308025 None None] PATH_INFO: `/` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:01:20.206516 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 733/1438] 127.0.0.1 () {54 vars in 901 bytes} [Mon Oct 10 12:01:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 10 12:01:20.214010 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99e826b-493e-402c-be80-0377650c08bc None None] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:01:20.214515 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99e826b-493e-402c-be80-0377650c08bc None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:01:20.214607 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99e826b-493e-402c-be80-0377650c08bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:01:20.306064 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.auth.core [None req-f99e826b-493e-402c-be80-0377650c08bc None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:01:20.307307 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-f99e826b-493e-402c-be80-0377650c08bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:01:20.309404 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:01:20.410463 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 706/1439] 127.0.0.1 () {58 vars in 998 bytes} [Mon Oct 10 12:01:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:01:20.562012 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7c8abe-4435-4027-89b1-f70959308025 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:01:20.563601 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-fd7c8abe-4435-4027-89b1-f70959308025 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:01:20.722107 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7c8abe-4435-4027-89b1-f70959308025 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:01:20.751855 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:01:20.841539 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddca4d71-d0f6-4ea7-90db-88b25764aaee service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f130bbb69e546ce99b4e14de81d7e54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad3772f7b6a47c28b7ba37a9316a3c1', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:01:20.842152 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddca4d71-d0f6-4ea7-90db-88b25764aaee service nova] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:01:20.842432 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddca4d71-d0f6-4ea7-90db-88b25764aaee service nova] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:01:20.842735 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddca4d71-d0f6-4ea7-90db-88b25764aaee service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:01:20.844345 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-ddca4d71-d0f6-4ea7-90db-88b25764aaee service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:01:20.943908 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddca4d71-d0f6-4ea7-90db-88b25764aaee service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:01:20.945555 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddca4d71-d0f6-4ea7-90db-88b25764aaee service nova] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:01:20.964174 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 734/1440] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 10 12:01:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:01:23.075387 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7b049a-7e16-4122-93eb-65f5698f1a61 None None] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:01:23.075998 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7b049a-7e16-4122-93eb-65f5698f1a61 None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:01:23.075998 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7b049a-7e16-4122-93eb-65f5698f1a61 None None] PATH_INFO: `/` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:01:23.077351 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 707/1441] 127.0.0.1 () {54 vars in 917 bytes} [Mon Oct 10 12:01:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 12:01:23.083065 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a7e209-68e9-447d-889e-bf00e70ea472 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:01:23.083601 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a7e209-68e9-447d-889e-bf00e70ea472 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:01:23.083601 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a7e209-68e9-447d-889e-bf00e70ea472 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:01:23.126616 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:01:23.154396 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-25a7e209-68e9-447d-889e-bf00e70ea472 None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:01:23.156330 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-25a7e209-68e9-447d-889e-bf00e70ea472 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:01:23.266085 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 735/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Oct 10 12:01:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:01:23.555267 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7b049a-7e16-4122-93eb-65f5698f1a61 None None] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:01:23.556801 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-4c7b049a-7e16-4122-93eb-65f5698f1a61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:01:23.667446 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7b049a-7e16-4122-93eb-65f5698f1a61 None None] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:01:23.701435 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:01:23.746311 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddfac46-909c-48e3-8de8-73b0634340bd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58b1c7c85302421cba823ace4201861f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad3772f7b6a47c28b7ba37a9316a3c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:01:23.746870 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddfac46-909c-48e3-8de8-73b0634340bd service neutron] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:01:23.747148 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddfac46-909c-48e3-8de8-73b0634340bd service neutron] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:01:23.747421 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddfac46-909c-48e3-8de8-73b0634340bd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:01:23.749034 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-8ddfac46-909c-48e3-8de8-73b0634340bd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:01:23.869382 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ddfac46-909c-48e3-8de8-73b0634340bd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:01:23.870869 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ddfac46-909c-48e3-8de8-73b0634340bd service neutron] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:01:23.890376 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 708/1443] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 10 12:01:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 10 12:01:26.391960 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb8ce5d-3ae5-4f10-8e75-14dba96ab251 None None] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:01:26.392546 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb8ce5d-3ae5-4f10-8e75-14dba96ab251 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:01:26.392546 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb8ce5d-3ae5-4f10-8e75-14dba96ab251 None None] PATH_INFO: `/` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:01:26.393888 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 736/1444] 127.0.0.1 () {54 vars in 917 bytes} [Mon Oct 10 12:01:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 12:01:26.399525 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114d798e-b146-4e3e-91bf-3d944466b984 None None] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:01:26.399967 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114d798e-b146-4e3e-91bf-3d944466b984 None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:01:26.400072 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114d798e-b146-4e3e-91bf-3d944466b984 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:01:26.473228 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.auth.core [None req-114d798e-b146-4e3e-91bf-3d944466b984 None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:01:26.474534 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-114d798e-b146-4e3e-91bf-3d944466b984 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:01:26.489626 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:01:26.578050 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 709/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Oct 10 12:01:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:01:26.864154 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb8ce5d-3ae5-4f10-8e75-14dba96ab251 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:01:26.865686 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-7eb8ce5d-3ae5-4f10-8e75-14dba96ab251 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:01:27.024660 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb8ce5d-3ae5-4f10-8e75-14dba96ab251 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:01:27.042597 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:01:27.111169 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91267ccf-4e6f-43d9-96fc-fbafaf55dafb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58b1c7c85302421cba823ace4201861f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad3772f7b6a47c28b7ba37a9316a3c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:01:27.111941 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91267ccf-4e6f-43d9-96fc-fbafaf55dafb service neutron] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:01:27.112272 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91267ccf-4e6f-43d9-96fc-fbafaf55dafb service neutron] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:01:27.112543 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91267ccf-4e6f-43d9-96fc-fbafaf55dafb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:01:27.114142 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-91267ccf-4e6f-43d9-96fc-fbafaf55dafb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:01:27.237857 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91267ccf-4e6f-43d9-96fc-fbafaf55dafb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:01:27.239332 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91267ccf-4e6f-43d9-96fc-fbafaf55dafb service neutron] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:01:27.258413 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 737/1446] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 10 12:01:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:01:29.889497 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291170a8-2949-4c56-b296-b5692a1fb1da None None] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:01:29.890063 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291170a8-2949-4c56-b296-b5692a1fb1da None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:01:29.890206 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291170a8-2949-4c56-b296-b5692a1fb1da None None] PATH_INFO: `/` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:01:29.891797 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 710/1447] 127.0.0.1 () {54 vars in 917 bytes} [Mon Oct 10 12:01:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 12:01:29.897294 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed35b78-d276-4e95-a7fe-7a084f360229 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:01:29.897765 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed35b78-d276-4e95-a7fe-7a084f360229 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:01:29.897871 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed35b78-d276-4e95-a7fe-7a084f360229 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:01:29.951330 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:01:29.969733 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-5ed35b78-d276-4e95-a7fe-7a084f360229 None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:01:29.971029 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-5ed35b78-d276-4e95-a7fe-7a084f360229 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:01:30.070855 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 738/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Oct 10 12:01:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 10 12:01:30.354363 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291170a8-2949-4c56-b296-b5692a1fb1da None None] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:01:30.355951 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-291170a8-2949-4c56-b296-b5692a1fb1da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:01:30.463636 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291170a8-2949-4c56-b296-b5692a1fb1da None None] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:01:30.504380 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:01:30.538908 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e560a75-b64a-4bf0-87f6-1e653c5deb8c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58b1c7c85302421cba823ace4201861f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad3772f7b6a47c28b7ba37a9316a3c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:01:30.539524 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e560a75-b64a-4bf0-87f6-1e653c5deb8c service neutron] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:01:30.539804 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e560a75-b64a-4bf0-87f6-1e653c5deb8c service neutron] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:01:30.540076 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e560a75-b64a-4bf0-87f6-1e653c5deb8c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:01:30.541738 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-6e560a75-b64a-4bf0-87f6-1e653c5deb8c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:01:30.655209 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e560a75-b64a-4bf0-87f6-1e653c5deb8c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:01:30.656648 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e560a75-b64a-4bf0-87f6-1e653c5deb8c service neutron] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:01:30.674970 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 711/1449] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 10 12:01:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:01:32.969440 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be77202-fa03-436b-a27d-68e016b09ee7 None None] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:01:32.970012 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be77202-fa03-436b-a27d-68e016b09ee7 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:01:32.970012 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be77202-fa03-436b-a27d-68e016b09ee7 None None] PATH_INFO: `/` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:01:32.971379 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 739/1450] 127.0.0.1 () {54 vars in 917 bytes} [Mon Oct 10 12:01:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 12:01:32.977527 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedb5f90-f0d6-423f-8dd8-18896510e966 None None] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:01:32.978023 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedb5f90-f0d6-423f-8dd8-18896510e966 None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:01:32.978117 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedb5f90-f0d6-423f-8dd8-18896510e966 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:01:33.047690 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.auth.core [None req-aedb5f90-f0d6-423f-8dd8-18896510e966 None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:01:33.048969 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-aedb5f90-f0d6-423f-8dd8-18896510e966 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:01:33.072536 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:01:33.155460 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 712/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Oct 10 12:01:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:01:33.442563 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be77202-fa03-436b-a27d-68e016b09ee7 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:01:33.444126 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-4be77202-fa03-436b-a27d-68e016b09ee7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:01:33.553141 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be77202-fa03-436b-a27d-68e016b09ee7 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:01:33.573568 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:01:33.631124 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1359a924-3523-46a8-bac7-bf4641e939d5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58b1c7c85302421cba823ace4201861f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad3772f7b6a47c28b7ba37a9316a3c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:01:33.631743 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1359a924-3523-46a8-bac7-bf4641e939d5 service neutron] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:01:33.632025 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1359a924-3523-46a8-bac7-bf4641e939d5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:01:33.632833 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1359a924-3523-46a8-bac7-bf4641e939d5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:01:33.634480 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-1359a924-3523-46a8-bac7-bf4641e939d5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:01:33.754964 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1359a924-3523-46a8-bac7-bf4641e939d5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:01:33.756413 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1359a924-3523-46a8-bac7-bf4641e939d5 service neutron] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:01:33.775571 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 740/1452] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 10 12:01:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:01:36.081852 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d84646-e3da-437c-957c-9b378906a104 None None] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:01:36.082437 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d84646-e3da-437c-957c-9b378906a104 None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:01:36.082437 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d84646-e3da-437c-957c-9b378906a104 None None] PATH_INFO: `/` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:01:36.083853 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 713/1453] 127.0.0.1 () {54 vars in 917 bytes} [Mon Oct 10 12:01:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 12:01:36.090082 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa349020-f48f-4918-85d6-837442b565d6 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:01:36.090536 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa349020-f48f-4918-85d6-837442b565d6 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:01:36.090657 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa349020-f48f-4918-85d6-837442b565d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:01:36.155736 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:01:36.162507 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-aa349020-f48f-4918-85d6-837442b565d6 None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:01:36.162507 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-aa349020-f48f-4918-85d6-837442b565d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:01:36.264046 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 741/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Oct 10 12:01:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:01:36.555309 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d84646-e3da-437c-957c-9b378906a104 None None] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:01:36.556850 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-b6d84646-e3da-437c-957c-9b378906a104 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:01:36.670577 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d84646-e3da-437c-957c-9b378906a104 None None] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:01:36.758813 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:01:36.784547 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3dc329-7f01-4cd3-8ecb-cdffd965f85e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58b1c7c85302421cba823ace4201861f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad3772f7b6a47c28b7ba37a9316a3c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:01:36.785160 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3dc329-7f01-4cd3-8ecb-cdffd965f85e service neutron] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:01:36.785461 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3dc329-7f01-4cd3-8ecb-cdffd965f85e service neutron] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:01:36.785718 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3dc329-7f01-4cd3-8ecb-cdffd965f85e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:01:36.787394 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-ed3dc329-7f01-4cd3-8ecb-cdffd965f85e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:01:36.898277 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3dc329-7f01-4cd3-8ecb-cdffd965f85e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:01:36.899750 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3dc329-7f01-4cd3-8ecb-cdffd965f85e service neutron] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:01:36.918913 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 714/1455] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 10 12:01:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:01:39.178681 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1fcd99-d1bb-4f27-b53c-3dcda896a6fe None None] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:01:39.179273 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1fcd99-d1bb-4f27-b53c-3dcda896a6fe None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:01:39.179273 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1fcd99-d1bb-4f27-b53c-3dcda896a6fe None None] PATH_INFO: `/` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:01:39.180940 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 742/1456] 127.0.0.1 () {54 vars in 917 bytes} [Mon Oct 10 12:01:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 12:01:39.186205 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c15aaf-272c-4910-9531-e81234d74658 None None] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:01:39.186624 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c15aaf-272c-4910-9531-e81234d74658 None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:01:39.186798 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c15aaf-272c-4910-9531-e81234d74658 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:01:39.253678 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.auth.core [None req-d5c15aaf-272c-4910-9531-e81234d74658 None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:01:39.254971 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-d5c15aaf-272c-4910-9531-e81234d74658 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:01:39.285606 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:01:39.353812 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 715/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Oct 10 12:01:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:01:39.640454 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1fcd99-d1bb-4f27-b53c-3dcda896a6fe None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:01:39.642015 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-ff1fcd99-d1bb-4f27-b53c-3dcda896a6fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:01:39.749237 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1fcd99-d1bb-4f27-b53c-3dcda896a6fe None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:01:39.778381 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:01:39.822915 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83046d4c-b752-4670-9d92-d362238a7099 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58b1c7c85302421cba823ace4201861f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad3772f7b6a47c28b7ba37a9316a3c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:01:39.823683 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83046d4c-b752-4670-9d92-d362238a7099 service neutron] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:01:39.824334 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83046d4c-b752-4670-9d92-d362238a7099 service neutron] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:01:39.824612 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83046d4c-b752-4670-9d92-d362238a7099 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:01:39.826230 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-83046d4c-b752-4670-9d92-d362238a7099 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:01:39.937799 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83046d4c-b752-4670-9d92-d362238a7099 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:01:39.939301 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83046d4c-b752-4670-9d92-d362238a7099 service neutron] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:01:39.957746 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 743/1458] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 10 12:01:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:01:42.192660 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fd96f2-df3d-4f26-9e48-7ce0abd5534e None None] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:01:42.193236 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fd96f2-df3d-4f26-9e48-7ce0abd5534e None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:01:42.193236 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fd96f2-df3d-4f26-9e48-7ce0abd5534e None None] PATH_INFO: `/` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:01:42.195140 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 716/1459] 127.0.0.1 () {54 vars in 917 bytes} [Mon Oct 10 12:01:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 12:01:42.200718 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb5a043-4504-4e67-9b40-b079d55c91a4 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:01:42.201144 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb5a043-4504-4e67-9b40-b079d55c91a4 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:01:42.201262 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb5a043-4504-4e67-9b40-b079d55c91a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:01:42.270037 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-7bb5a043-4504-4e67-9b40-b079d55c91a4 None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:01:42.271393 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-7bb5a043-4504-4e67-9b40-b079d55c91a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:01:42.281451 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:01:42.371000 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 744/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Oct 10 12:01:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:01:42.658070 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fd96f2-df3d-4f26-9e48-7ce0abd5534e None None] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:01:42.659654 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-b9fd96f2-df3d-4f26-9e48-7ce0abd5534e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:01:42.767057 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fd96f2-df3d-4f26-9e48-7ce0abd5534e None None] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:01:42.824471 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:01:42.842832 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11c7a7e-ff2f-403c-b9bb-5dc2afdbfd5b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58b1c7c85302421cba823ace4201861f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad3772f7b6a47c28b7ba37a9316a3c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:01:42.843450 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11c7a7e-ff2f-403c-b9bb-5dc2afdbfd5b service neutron] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:01:42.843725 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11c7a7e-ff2f-403c-b9bb-5dc2afdbfd5b service neutron] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:01:42.843998 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11c7a7e-ff2f-403c-b9bb-5dc2afdbfd5b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:01:42.845619 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-c11c7a7e-ff2f-403c-b9bb-5dc2afdbfd5b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:01:42.957856 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11c7a7e-ff2f-403c-b9bb-5dc2afdbfd5b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:01:42.959423 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11c7a7e-ff2f-403c-b9bb-5dc2afdbfd5b service neutron] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:01:42.978925 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 717/1461] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 10 12:01:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:01:45.227484 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef3e37a-ff81-4671-8312-21d7d5822872 None None] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:01:45.228041 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef3e37a-ff81-4671-8312-21d7d5822872 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:01:45.228041 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef3e37a-ff81-4671-8312-21d7d5822872 None None] PATH_INFO: `/` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:01:45.229238 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 745/1462] 127.0.0.1 () {54 vars in 917 bytes} [Mon Oct 10 12:01:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 12:01:45.234714 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d82d50a-b103-491a-93f6-a89cfaa4d1b3 None None] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:01:45.235226 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d82d50a-b103-491a-93f6-a89cfaa4d1b3 None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:01:45.235322 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d82d50a-b103-491a-93f6-a89cfaa4d1b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:01:45.302337 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.auth.core [None req-9d82d50a-b103-491a-93f6-a89cfaa4d1b3 None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:01:45.303592 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-9d82d50a-b103-491a-93f6-a89cfaa4d1b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:01:45.342232 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:01:45.400789 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 718/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Oct 10 12:01:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:01:45.689004 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef3e37a-ff81-4671-8312-21d7d5822872 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:01:45.690549 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-aef3e37a-ff81-4671-8312-21d7d5822872 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:01:45.795419 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef3e37a-ff81-4671-8312-21d7d5822872 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:01:45.833343 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:01:45.872686 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441cfd79-da7e-48ee-976e-949fb8d3141d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58b1c7c85302421cba823ace4201861f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad3772f7b6a47c28b7ba37a9316a3c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:01:45.873299 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441cfd79-da7e-48ee-976e-949fb8d3141d service neutron] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:01:45.873577 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441cfd79-da7e-48ee-976e-949fb8d3141d service neutron] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:01:45.873850 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441cfd79-da7e-48ee-976e-949fb8d3141d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:01:45.875493 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-441cfd79-da7e-48ee-976e-949fb8d3141d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:01:45.984603 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441cfd79-da7e-48ee-976e-949fb8d3141d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:01:45.986040 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441cfd79-da7e-48ee-976e-949fb8d3141d service neutron] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:01:46.005439 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 746/1464] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 10 12:01:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:01:48.220352 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7005434-d08a-431c-a59b-f56b376d8ff2 None None] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:01:48.220898 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7005434-d08a-431c-a59b-f56b376d8ff2 None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:01:48.220898 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7005434-d08a-431c-a59b-f56b376d8ff2 None None] PATH_INFO: `/` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:01:48.222265 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 719/1465] 127.0.0.1 () {54 vars in 917 bytes} [Mon Oct 10 12:01:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 10 12:01:48.228019 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec65d1d-89fd-48a8-af2e-d404457aed14 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:01:48.228463 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec65d1d-89fd-48a8-af2e-d404457aed14 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:01:48.228558 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec65d1d-89fd-48a8-af2e-d404457aed14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:01:48.308895 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-9ec65d1d-89fd-48a8-af2e-d404457aed14 None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:01:48.311043 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-9ec65d1d-89fd-48a8-af2e-d404457aed14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:01:48.343059 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:01:48.430240 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 747/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Oct 10 12:01:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 10 12:01:48.713804 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7005434-d08a-431c-a59b-f56b376d8ff2 None None] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:01:48.715429 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-e7005434-d08a-431c-a59b-f56b376d8ff2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:01:48.827040 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7005434-d08a-431c-a59b-f56b376d8ff2 None None] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:01:48.903085 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:01:48.909636 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f671ef-6c26-40f4-a460-0a084b97ed26 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58b1c7c85302421cba823ace4201861f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad3772f7b6a47c28b7ba37a9316a3c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:01:48.910294 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f671ef-6c26-40f4-a460-0a084b97ed26 service neutron] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:01:48.910572 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f671ef-6c26-40f4-a460-0a084b97ed26 service neutron] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:01:48.910881 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f671ef-6c26-40f4-a460-0a084b97ed26 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:01:48.912507 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-a6f671ef-6c26-40f4-a460-0a084b97ed26 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:01:49.022019 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f671ef-6c26-40f4-a460-0a084b97ed26 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:01:49.023964 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f671ef-6c26-40f4-a460-0a084b97ed26 service neutron] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:01:49.042980 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 720/1467] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 10 12:01:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:01:51.207393 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b8c3b9-048e-4757-a5a6-0afe3167c33a None None] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:01:51.207978 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b8c3b9-048e-4757-a5a6-0afe3167c33a None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:01:51.207978 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b8c3b9-048e-4757-a5a6-0afe3167c33a None None] PATH_INFO: `/` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:01:51.209280 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 748/1468] 127.0.0.1 () {54 vars in 917 bytes} [Mon Oct 10 12:01:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 12:01:51.214746 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7392ecca-fbbd-4546-a778-ce2dd8380591 None None] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:01:51.215200 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7392ecca-fbbd-4546-a778-ce2dd8380591 None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:01:51.215316 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7392ecca-fbbd-4546-a778-ce2dd8380591 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:01:51.281220 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.auth.core [None req-7392ecca-fbbd-4546-a778-ce2dd8380591 None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:01:51.282439 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-7392ecca-fbbd-4546-a778-ce2dd8380591 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:01:51.330397 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:01:51.379680 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 721/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Oct 10 12:01:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:01:51.660595 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b8c3b9-048e-4757-a5a6-0afe3167c33a None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:01:51.662138 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-56b8c3b9-048e-4757-a5a6-0afe3167c33a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:01:51.773183 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b8c3b9-048e-4757-a5a6-0afe3167c33a None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:01:51.822829 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:01:51.850231 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3922a465-aa38-4424-ba18-d110275ab0a4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58b1c7c85302421cba823ace4201861f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad3772f7b6a47c28b7ba37a9316a3c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:01:51.851478 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3922a465-aa38-4424-ba18-d110275ab0a4 service neutron] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:01:51.851992 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3922a465-aa38-4424-ba18-d110275ab0a4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:01:51.852291 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3922a465-aa38-4424-ba18-d110275ab0a4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:01:51.854423 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-3922a465-aa38-4424-ba18-d110275ab0a4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:01:51.964164 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3922a465-aa38-4424-ba18-d110275ab0a4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:01:51.965598 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3922a465-aa38-4424-ba18-d110275ab0a4 service neutron] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:01:51.984498 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 749/1470] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 10 12:01:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:02:01.378033 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=22 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:02:01.982133 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=66 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:02:12.053763 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8779215b-b55c-497b-8b2e-7e0752730f1f None None] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:02:12.054306 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8779215b-b55c-497b-8b2e-7e0752730f1f None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:02:12.054306 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8779215b-b55c-497b-8b2e-7e0752730f1f None None] PATH_INFO: `/` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:02:12.055730 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 722/1471] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 10 12:02:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 12:02:12.061281 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462339e0-c4d9-4b51-af61-8437a4d3b891 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:02:12.061751 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462339e0-c4d9-4b51-af61-8437a4d3b891 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:02:12.061875 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462339e0-c4d9-4b51-af61-8437a4d3b891 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:02:12.129014 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-462339e0-c4d9-4b51-af61-8437a4d3b891 None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:02:12.130234 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-462339e0-c4d9-4b51-af61-8437a4d3b891 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:02:12.228450 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 750/1472] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 10 12:02:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:02:12.525378 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d3fd21-823f-4eab-a75c-d68482e21c09 None None] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:02:12.525665 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d3fd21-823f-4eab-a75c-d68482e21c09 None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:02:12.525949 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d3fd21-823f-4eab-a75c-d68482e21c09 None None] PATH_INFO: `/` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:02:12.527329 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 723/1473] 10.222.0.46 () {54 vars in 962 bytes} [Mon Oct 10 12:02:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 12:02:12.532591 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a006df39-ae61-4add-a904-46951ed0e42e None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:02:12.532883 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a006df39-ae61-4add-a904-46951ed0e42e None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:02:12.533162 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a006df39-ae61-4add-a904-46951ed0e42e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:02:12.609647 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-a006df39-ae61-4add-a904-46951ed0e42e None None] MFA Rules not processed for user `124f4aa8ab244ca2aadf4f2614df51e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:02:12.611413 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-a006df39-ae61-4add-a904-46951ed0e42e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:02:12.631821 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:02:12.686331 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 751/1474] 10.222.0.46 () {58 vars in 1059 bytes} [Mon Oct 10 12:02:12 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:02:12.694705 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d3fd21-823f-4eab-a75c-d68482e21c09 None None] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:02:12.696238 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-d7d3fd21-823f-4eab-a75c-d68482e21c09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:02:12.801348 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d3fd21-823f-4eab-a75c-d68482e21c09 None None] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:02:12.877280 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b4bd08-c153-40dd-bb34-649b4f3bf8e2 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '124f4aa8ab244ca2aadf4f2614df51e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad3772f7b6a47c28b7ba37a9316a3c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:02:12.877887 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b4bd08-c153-40dd-bb34-649b4f3bf8e2 service glance] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:02:12.878165 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b4bd08-c153-40dd-bb34-649b4f3bf8e2 service glance] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:02:12.878506 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b4bd08-c153-40dd-bb34-649b4f3bf8e2 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:02:12.880132 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-63b4bd08-c153-40dd-bb34-649b4f3bf8e2 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:02:12.934977 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:02:12.989761 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b4bd08-c153-40dd-bb34-649b4f3bf8e2 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:02:12.991758 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b4bd08-c153-40dd-bb34-649b4f3bf8e2 service glance] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:02:13.010748 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 724/1475] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 10 12:02:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:02:15.296544 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8a4b47-598b-4d51-a22c-430f47269b58 None None] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:02:15.297097 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8a4b47-598b-4d51-a22c-430f47269b58 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:02:15.297097 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8a4b47-598b-4d51-a22c-430f47269b58 None None] PATH_INFO: `/` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:02:15.298369 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 752/1476] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 10 12:02:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 12:02:15.303755 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bcd631-8bba-4b32-bd1c-1807283ac638 None None] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:02:15.304216 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bcd631-8bba-4b32-bd1c-1807283ac638 None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:02:15.304311 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bcd631-8bba-4b32-bd1c-1807283ac638 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:02:15.370648 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.auth.core [None req-a9bcd631-8bba-4b32-bd1c-1807283ac638 None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:02:15.371919 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-a9bcd631-8bba-4b32-bd1c-1807283ac638 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:02:15.469927 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 725/1477] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 10 12:02:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:02:15.746729 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8a4b47-598b-4d51-a22c-430f47269b58 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:02:15.748317 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-eb8a4b47-598b-4d51-a22c-430f47269b58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:02:15.873434 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8a4b47-598b-4d51-a22c-430f47269b58 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:02:15.924343 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:02:15.970517 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1140672-038d-48f4-a7f8-a1835faac8c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:02:15.971290 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1140672-038d-48f4-a7f8-a1835faac8c2 admin admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:02:15.971572 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1140672-038d-48f4-a7f8-a1835faac8c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:02:15.971845 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1140672-038d-48f4-a7f8-a1835faac8c2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:02:15.972485 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1140672-038d-48f4-a7f8-a1835faac8c2 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:02:15.974248 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: WARNING py.warnings [None req-b1140672-038d-48f4-a7f8-a1835faac8c2 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 10 12:02:15.974248 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: warnings.warn(msg) Oct 10 12:02:15.974248 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: Oct 10 12:02:15.975432 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1140672-038d-48f4-a7f8-a1835faac8c2 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:02:15.995387 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 753/1478] 10.222.0.46 () {56 vars in 1108 bytes} [Mon Oct 10 12:02:15 2022] GET /identity/v3/projects => generated 2553 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 12:02:18.194720 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227173f8-9283-488b-9d1f-856ce031deca None None] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:02:18.195270 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227173f8-9283-488b-9d1f-856ce031deca None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:02:18.195270 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227173f8-9283-488b-9d1f-856ce031deca None None] PATH_INFO: `/` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:02:18.196615 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 726/1479] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 10 12:02:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 12:02:18.202047 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bdd25b-846c-4228-a5e1-2f68773a924e None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:02:18.202495 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bdd25b-846c-4228-a5e1-2f68773a924e None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:02:18.202640 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bdd25b-846c-4228-a5e1-2f68773a924e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:02:18.271793 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-90bdd25b-846c-4228-a5e1-2f68773a924e None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:02:18.273047 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-90bdd25b-846c-4228-a5e1-2f68773a924e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:02:18.371451 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 754/1480] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 10 12:02:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:02:18.663664 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227173f8-9283-488b-9d1f-856ce031deca None None] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:02:18.665221 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-227173f8-9283-488b-9d1f-856ce031deca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:02:18.675092 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:02:18.814996 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227173f8-9283-488b-9d1f-856ce031deca None None] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:02:18.895266 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:02:18.930081 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bff1544-73a4-44b5-b644-ce6c2bb58ff9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f130bbb69e546ce99b4e14de81d7e54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad3772f7b6a47c28b7ba37a9316a3c1', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:02:18.930739 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bff1544-73a4-44b5-b644-ce6c2bb58ff9 service nova] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:02:18.931019 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bff1544-73a4-44b5-b644-ce6c2bb58ff9 service nova] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:02:18.931290 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bff1544-73a4-44b5-b644-ce6c2bb58ff9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:02:18.932871 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-0bff1544-73a4-44b5-b644-ce6c2bb58ff9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:02:19.022883 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bff1544-73a4-44b5-b644-ce6c2bb58ff9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:02:19.024493 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bff1544-73a4-44b5-b644-ce6c2bb58ff9 service nova] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:02:19.043403 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 727/1481] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 10 12:02:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:02:21.403532 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7197c5c6-70a1-48e7-b69f-8288990db113 None None] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:02:21.403532 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7197c5c6-70a1-48e7-b69f-8288990db113 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:02:21.403532 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7197c5c6-70a1-48e7-b69f-8288990db113 None None] PATH_INFO: `/` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:02:21.404980 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 755/1482] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 10 12:02:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 12:02:21.410609 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c505828d-19d1-4992-bf52-ebf227867d2e None None] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:02:21.411135 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c505828d-19d1-4992-bf52-ebf227867d2e None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:02:21.411245 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c505828d-19d1-4992-bf52-ebf227867d2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:02:21.480147 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.auth.core [None req-c505828d-19d1-4992-bf52-ebf227867d2e None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:02:21.481378 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-c505828d-19d1-4992-bf52-ebf227867d2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:02:21.529101 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:02:21.582620 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 728/1483] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 10 12:02:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:02:21.870367 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7197c5c6-70a1-48e7-b69f-8288990db113 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:02:21.872061 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-7197c5c6-70a1-48e7-b69f-8288990db113 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:02:21.899889 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:02:21.980504 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7197c5c6-70a1-48e7-b69f-8288990db113 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:02:22.053763 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2760d65b-54bf-4273-8ea7-aa3c1801d4a4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58b1c7c85302421cba823ace4201861f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad3772f7b6a47c28b7ba37a9316a3c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:02:22.054371 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2760d65b-54bf-4273-8ea7-aa3c1801d4a4 service neutron] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:02:22.054651 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2760d65b-54bf-4273-8ea7-aa3c1801d4a4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:02:22.054969 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2760d65b-54bf-4273-8ea7-aa3c1801d4a4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:02:22.056571 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-2760d65b-54bf-4273-8ea7-aa3c1801d4a4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:02:22.130111 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:02:22.164836 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2760d65b-54bf-4273-8ea7-aa3c1801d4a4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:02:22.166265 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2760d65b-54bf-4273-8ea7-aa3c1801d4a4 service neutron] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:02:22.184236 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 756/1484] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 10 12:02:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:02:24.430809 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff35a0da-9f0d-4114-b57d-431f55963d1a None None] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:02:24.431403 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff35a0da-9f0d-4114-b57d-431f55963d1a None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:02:24.431403 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff35a0da-9f0d-4114-b57d-431f55963d1a None None] PATH_INFO: `/` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:02:24.432757 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 729/1485] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 10 12:02:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 12:02:24.438291 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068dbffe-9bd7-4268-a58e-f943a401724c None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:02:24.438791 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068dbffe-9bd7-4268-a58e-f943a401724c None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:02:24.438899 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068dbffe-9bd7-4268-a58e-f943a401724c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:02:24.510586 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-068dbffe-9bd7-4268-a58e-f943a401724c None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:02:24.511843 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-068dbffe-9bd7-4268-a58e-f943a401724c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:02:24.609369 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 757/1486] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 10 12:02:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 10 12:02:24.901018 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff35a0da-9f0d-4114-b57d-431f55963d1a None None] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:02:24.902556 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-ff35a0da-9f0d-4114-b57d-431f55963d1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:02:25.009053 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff35a0da-9f0d-4114-b57d-431f55963d1a None None] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:02:25.085589 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:02:25.087121 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d081ec-080a-488d-9a9d-0adab9605f2b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58b1c7c85302421cba823ace4201861f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad3772f7b6a47c28b7ba37a9316a3c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:02:25.087736 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d081ec-080a-488d-9a9d-0adab9605f2b service neutron] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:02:25.088019 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d081ec-080a-488d-9a9d-0adab9605f2b service neutron] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:02:25.088292 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d081ec-080a-488d-9a9d-0adab9605f2b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:02:25.089917 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-c7d081ec-080a-488d-9a9d-0adab9605f2b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:02:25.203417 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d081ec-080a-488d-9a9d-0adab9605f2b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:02:25.204755 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d081ec-080a-488d-9a9d-0adab9605f2b service neutron] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:02:25.224060 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 730/1487] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 10 12:02:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:02:27.537637 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9eb15c-c7bb-4a37-970e-8c4cd7ea3cb1 None None] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:02:27.538197 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9eb15c-c7bb-4a37-970e-8c4cd7ea3cb1 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:02:27.538197 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9eb15c-c7bb-4a37-970e-8c4cd7ea3cb1 None None] PATH_INFO: `/` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:02:27.539555 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 758/1488] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 10 12:02:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 12:02:27.544984 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7c8466-1c4f-407b-9a15-cbad79d44fa8 None None] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:02:27.545458 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7c8466-1c4f-407b-9a15-cbad79d44fa8 None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:02:27.545559 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7c8466-1c4f-407b-9a15-cbad79d44fa8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:02:27.612626 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.auth.core [None req-6a7c8466-1c4f-407b-9a15-cbad79d44fa8 None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:02:27.613856 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-6a7c8466-1c4f-407b-9a15-cbad79d44fa8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:02:27.667636 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:02:27.711859 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 731/1489] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 10 12:02:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 10 12:02:28.009108 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9eb15c-c7bb-4a37-970e-8c4cd7ea3cb1 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:02:28.010728 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-ca9eb15c-c7bb-4a37-970e-8c4cd7ea3cb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:02:28.047942 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:02:28.116939 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9eb15c-c7bb-4a37-970e-8c4cd7ea3cb1 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:02:28.191229 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a54bb1-3cf2-4f79-8090-3d01e6b9a89a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58b1c7c85302421cba823ace4201861f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad3772f7b6a47c28b7ba37a9316a3c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:02:28.195229 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a54bb1-3cf2-4f79-8090-3d01e6b9a89a service neutron] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:02:28.195229 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a54bb1-3cf2-4f79-8090-3d01e6b9a89a service neutron] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:02:28.195229 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a54bb1-3cf2-4f79-8090-3d01e6b9a89a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:02:28.195229 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-90a54bb1-3cf2-4f79-8090-3d01e6b9a89a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:02:28.283250 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:02:28.315888 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a54bb1-3cf2-4f79-8090-3d01e6b9a89a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:02:28.317354 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a54bb1-3cf2-4f79-8090-3d01e6b9a89a service neutron] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:02:28.335807 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 759/1490] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 10 12:02:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:02:30.851482 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea18a16-d70e-46f5-8763-aea643af4883 None None] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:02:30.852055 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea18a16-d70e-46f5-8763-aea643af4883 None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:02:30.852055 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea18a16-d70e-46f5-8763-aea643af4883 None None] PATH_INFO: `/` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:02:30.853402 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 732/1491] 10.222.0.46 () {54 vars in 927 bytes} [Mon Oct 10 12:02:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 12:02:30.858734 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ee2234-587f-4d27-a2ff-fd569b7aa75a None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:02:30.859203 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ee2234-587f-4d27-a2ff-fd569b7aa75a None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:02:30.859309 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ee2234-587f-4d27-a2ff-fd569b7aa75a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:02:30.927049 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-f6ee2234-587f-4d27-a2ff-fd569b7aa75a None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:02:30.928761 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-f6ee2234-587f-4d27-a2ff-fd569b7aa75a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:02:31.026507 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 760/1492] 10.222.0.46 () {58 vars in 1024 bytes} [Mon Oct 10 12:02:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:02:31.315272 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea18a16-d70e-46f5-8763-aea643af4883 None None] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:02:31.316822 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-7ea18a16-d70e-46f5-8763-aea643af4883 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:02:31.430117 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea18a16-d70e-46f5-8763-aea643af4883 None None] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:02:31.505572 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928a436f-30ca-4328-9745-842f93d5fc2c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58b1c7c85302421cba823ace4201861f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad3772f7b6a47c28b7ba37a9316a3c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:02:31.506181 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928a436f-30ca-4328-9745-842f93d5fc2c service neutron] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:02:31.506469 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928a436f-30ca-4328-9745-842f93d5fc2c service neutron] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:02:31.506780 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928a436f-30ca-4328-9745-842f93d5fc2c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:02:31.508396 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-928a436f-30ca-4328-9745-842f93d5fc2c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:02:31.517228 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:02:31.620924 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928a436f-30ca-4328-9745-842f93d5fc2c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:02:31.622277 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928a436f-30ca-4328-9745-842f93d5fc2c service neutron] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:02:31.640332 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 733/1493] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 10 12:02:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:02:41.024972 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=82 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:02:41.638039 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=50 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:04:06.203686 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1df677-d473-462c-9501-8984340221df None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:04:06.204340 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1df677-d473-462c-9501-8984340221df None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:04:06.204340 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1df677-d473-462c-9501-8984340221df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:04:06.292342 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-5f1df677-d473-462c-9501-8984340221df None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:04:06.293567 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-5f1df677-d473-462c-9501-8984340221df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:04:06.403136 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 761/1494] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:04:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:04:06.411645 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca82bb25-d2ce-44c0-b505-2366c6d4313d None None] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:04:06.412195 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca82bb25-d2ce-44c0-b505-2366c6d4313d None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:04:06.412195 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca82bb25-d2ce-44c0-b505-2366c6d4313d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:04:06.433092 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.auth.core [None req-ca82bb25-d2ce-44c0-b505-2366c6d4313d None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:04:06.434314 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-ca82bb25-d2ce-44c0-b505-2366c6d4313d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:04:06.443262 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 734/1495] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:04:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:04:06.448410 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1df677-d473-462c-9501-8984340221df None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:04:06.458334 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1df677-d473-462c-9501-8984340221df None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:04:06.463707 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774369f8-6ee4-452b-80ec-ca4849d5d4b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:04:06.464958 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774369f8-6ee4-452b-80ec-ca4849d5d4b1 admin admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:04:06.465248 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774369f8-6ee4-452b-80ec-ca4849d5d4b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:04:06.465675 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774369f8-6ee4-452b-80ec-ca4849d5d4b1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:04:06.466345 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774369f8-6ee4-452b-80ec-ca4849d5d4b1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 12:04:06.466708 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774369f8-6ee4-452b-80ec-ca4849d5d4b1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:04:06.468109 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: WARNING py.warnings [None req-774369f8-6ee4-452b-80ec-ca4849d5d4b1 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 10 12:04:06.468109 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: warnings.warn(msg) Oct 10 12:04:06.468109 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: Oct 10 12:04:06.469008 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774369f8-6ee4-452b-80ec-ca4849d5d4b1 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:04:06.481763 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 762/1496] 10.222.0.46 () {58 vars in 1153 bytes} [Mon Oct 10 12:04:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 12:04:06.490925 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca82bb25-d2ce-44c0-b505-2366c6d4313d None None] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:04:06.500311 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca82bb25-d2ce-44c0-b505-2366c6d4313d None None] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:04:06.505044 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416b3d2e-be8d-41c1-87e4-e4d4d798eefa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:04:06.505648 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416b3d2e-be8d-41c1-87e4-e4d4d798eefa admin admin] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:04:06.505930 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416b3d2e-be8d-41c1-87e4-e4d4d798eefa admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:04:06.506204 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416b3d2e-be8d-41c1-87e4-e4d4d798eefa admin admin] PATH_INFO: `/v3/projects` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:04:06.506895 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416b3d2e-be8d-41c1-87e4-e4d4d798eefa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:04:06.508646 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: WARNING py.warnings [None req-416b3d2e-be8d-41c1-87e4-e4d4d798eefa admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 10 12:04:06.508646 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: warnings.warn(msg) Oct 10 12:04:06.508646 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: Oct 10 12:04:06.509829 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416b3d2e-be8d-41c1-87e4-e4d4d798eefa admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:04:06.634360 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 735/1497] 10.222.0.46 () {60 vars in 1153 bytes} [Mon Oct 10 12:04:06 2022] POST /identity/v3/projects => generated 379 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 12:04:06.638954 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774369f8-6ee4-452b-80ec-ca4849d5d4b1 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:04:06.649124 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774369f8-6ee4-452b-80ec-ca4849d5d4b1 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:04:06.703971 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:04:06.713628 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9cf7bd-fc00-40c7-8697-ca296a37d710 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:04:06.714365 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9cf7bd-fc00-40c7-8697-ca296a37d710 admin admin] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:04:06.714781 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9cf7bd-fc00-40c7-8697-ca296a37d710 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:04:06.715153 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9cf7bd-fc00-40c7-8697-ca296a37d710 admin admin] PATH_INFO: `/v3/users` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:04:06.716081 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9cf7bd-fc00-40c7-8697-ca296a37d710 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:04:06.717497 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: WARNING py.warnings [None req-1b9cf7bd-fc00-40c7-8697-ca296a37d710 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 10 12:04:06.717497 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: warnings.warn(msg) Oct 10 12:04:06.717497 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: Oct 10 12:04:06.719320 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9cf7bd-fc00-40c7-8697-ca296a37d710 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:04:06.856825 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 763/1498] 10.222.0.46 () {60 vars in 1144 bytes} [Mon Oct 10 12:04:06 2022] POST /identity/v3/users => generated 290 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 12:04:06.862245 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416b3d2e-be8d-41c1-87e4-e4d4d798eefa admin admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:04:06.872474 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416b3d2e-be8d-41c1-87e4-e4d4d798eefa admin admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:04:06.877331 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39163d3-b6a6-4040-8e11-72551c6c2b1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:04:06.877947 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39163d3-b6a6-4040-8e11-72551c6c2b1b admin admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:04:06.878228 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39163d3-b6a6-4040-8e11-72551c6c2b1b admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:04:06.878499 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39163d3-b6a6-4040-8e11-72551c6c2b1b admin admin] PATH_INFO: `/v3/roles` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:04:06.879200 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39163d3-b6a6-4040-8e11-72551c6c2b1b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:04:06.880217 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: WARNING py.warnings [None req-c39163d3-b6a6-4040-8e11-72551c6c2b1b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 10 12:04:06.880217 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: warnings.warn(msg) Oct 10 12:04:06.880217 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: Oct 10 12:04:06.881095 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39163d3-b6a6-4040-8e11-72551c6c2b1b admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:04:06.895764 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 736/1499] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 10 12:04:06 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 12:04:06.900767 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9cf7bd-fc00-40c7-8697-ca296a37d710 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:04:06.914284 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9cf7bd-fc00-40c7-8697-ca296a37d710 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:04:06.919215 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904a2624-9315-4a51-9a16-65c9016fdb97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:04:06.919836 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904a2624-9315-4a51-9a16-65c9016fdb97 admin admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:04:06.920120 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904a2624-9315-4a51-9a16-65c9016fdb97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:04:06.920391 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904a2624-9315-4a51-9a16-65c9016fdb97 admin admin] PATH_INFO: `/v3/roles` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:04:06.921035 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904a2624-9315-4a51-9a16-65c9016fdb97 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:04:06.922085 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: WARNING py.warnings [None req-904a2624-9315-4a51-9a16-65c9016fdb97 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 10 12:04:06.922085 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: warnings.warn(msg) Oct 10 12:04:06.922085 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: Oct 10 12:04:06.923027 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904a2624-9315-4a51-9a16-65c9016fdb97 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:04:06.937100 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 764/1500] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 10 12:04:06 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 12:04:06.941581 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39163d3-b6a6-4040-8e11-72551c6c2b1b admin admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:04:06.951704 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39163d3-b6a6-4040-8e11-72551c6c2b1b admin admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:04:06.956371 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de41ca30-9ce3-4aa7-881a-b252e01e8e0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:04:06.957054 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de41ca30-9ce3-4aa7-881a-b252e01e8e0c admin admin] REQUEST_METHOD: `PUT` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:04:06.957341 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de41ca30-9ce3-4aa7-881a-b252e01e8e0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:04:06.957621 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de41ca30-9ce3-4aa7-881a-b252e01e8e0c admin admin] PATH_INFO: `/v3/projects/b093cd038d8a44499dfffe2333d0a537/users/82dfb2823cfb4273a94909f2c485ac95/roles/b2cb2a6e818e47338ce071e6fedfc673` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:04:06.976561 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de41ca30-9ce3-4aa7-881a-b252e01e8e0c admin admin] RBAC: Authorizing `identity:create_grant(project_id=b093cd038d8a44499dfffe2333d0a537, user_id=82dfb2823cfb4273a94909f2c485ac95, role_id=b2cb2a6e818e47338ce071e6fedfc673)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:04:06.977580 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: WARNING py.warnings [None req-de41ca30-9ce3-4aa7-881a-b252e01e8e0c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 10 12:04:06.977580 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: warnings.warn(msg) Oct 10 12:04:06.977580 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: Oct 10 12:04:06.979553 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de41ca30-9ce3-4aa7-881a-b252e01e8e0c admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:04:07.107759 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 737/1501] 10.222.0.46 () {60 vars in 1483 bytes} [Mon Oct 10 12:04:06 2022] PUT /identity/v3/projects/b093cd038d8a44499dfffe2333d0a537/users/82dfb2823cfb4273a94909f2c485ac95/roles/b2cb2a6e818e47338ce071e6fedfc673 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:04:07.120692 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d20d5d1-a253-4dde-8646-7e4cd7b52eef None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:04:07.121075 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d20d5d1-a253-4dde-8646-7e4cd7b52eef None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:04:07.121354 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d20d5d1-a253-4dde-8646-7e4cd7b52eef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:04:07.151054 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-4d20d5d1-a253-4dde-8646-7e4cd7b52eef None None] MFA Rules not processed for user `82dfb2823cfb4273a94909f2c485ac95`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:04:07.152266 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-4d20d5d1-a253-4dde-8646-7e4cd7b52eef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:04:07.216528 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 765/1502] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:04:07 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:04:07.230192 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2503b89-2b1b-4484-adf8-22cb415be1b3 None None] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:04:07.230420 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2503b89-2b1b-4484-adf8-22cb415be1b3 None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:04:07.230842 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2503b89-2b1b-4484-adf8-22cb415be1b3 None None] PATH_INFO: `/` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:04:07.232193 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 738/1503] 10.222.0.46 () {54 vars in 963 bytes} [Mon Oct 10 12:04:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 12:04:07.237304 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089cfd0e-2a76-4369-9f8e-1a8f432aea56 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:04:07.237595 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089cfd0e-2a76-4369-9f8e-1a8f432aea56 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:04:07.237876 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089cfd0e-2a76-4369-9f8e-1a8f432aea56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:04:07.297843 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-089cfd0e-2a76-4369-9f8e-1a8f432aea56 None None] MFA Rules not processed for user `d3ae093470484b93b23dcfae1657df9e`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:04:07.299066 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-089cfd0e-2a76-4369-9f8e-1a8f432aea56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:04:07.377364 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 766/1504] 10.222.0.46 () {58 vars in 1060 bytes} [Mon Oct 10 12:04:07 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 10 12:04:07.385795 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2503b89-2b1b-4484-adf8-22cb415be1b3 None None] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:04:07.397468 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2503b89-2b1b-4484-adf8-22cb415be1b3 None None] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:04:07.401045 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540cc5de-b78c-4ecb-ad24-abf48033f2c4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3ae093470484b93b23dcfae1657df9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad3772f7b6a47c28b7ba37a9316a3c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:04:07.401642 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540cc5de-b78c-4ecb-ad24-abf48033f2c4 service cinder] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:04:07.401923 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540cc5de-b78c-4ecb-ad24-abf48033f2c4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:04:07.402201 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540cc5de-b78c-4ecb-ad24-abf48033f2c4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:04:07.410649 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540cc5de-b78c-4ecb-ad24-abf48033f2c4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:04:07.412126 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540cc5de-b78c-4ecb-ad24-abf48033f2c4 service cinder] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:04:07.421158 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 739/1505] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 10 12:04:07 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:04:07.468489 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b96da5-b810-4f27-8f14-1c0c786962f7 None None] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:04:07.468780 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b96da5-b810-4f27-8f14-1c0c786962f7 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:04:07.469048 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b96da5-b810-4f27-8f14-1c0c786962f7 None None] PATH_INFO: `/` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:04:07.470303 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 767/1506] 10.222.0.46 () {52 vars in 830 bytes} [Mon Oct 10 12:04:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 12:04:07.474044 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540cc5de-b78c-4ecb-ad24-abf48033f2c4 service cinder] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:04:07.536271 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540cc5de-b78c-4ecb-ad24-abf48033f2c4 service cinder] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:04:07.540906 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6801c2e9-872e-4f85-a0cb-d7c77a631f93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:04:07.541531 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6801c2e9-872e-4f85-a0cb-d7c77a631f93 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:04:07.541810 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6801c2e9-872e-4f85-a0cb-d7c77a631f93 admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:04:07.542082 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6801c2e9-872e-4f85-a0cb-d7c77a631f93 admin admin] PATH_INFO: `/v3/users/82dfb2823cfb4273a94909f2c485ac95` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:04:07.543233 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6801c2e9-872e-4f85-a0cb-d7c77a631f93 admin admin] RBAC: Authorizing `identity:delete_user(user_id=82dfb2823cfb4273a94909f2c485ac95)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:04:07.544250 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: WARNING py.warnings [None req-6801c2e9-872e-4f85-a0cb-d7c77a631f93 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 10 12:04:07.544250 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: warnings.warn(msg) Oct 10 12:04:07.544250 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: Oct 10 12:04:07.545494 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6801c2e9-872e-4f85-a0cb-d7c77a631f93 admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:04:07.792421 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-6801c2e9-872e-4f85-a0cb-d7c77a631f93 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '82dfb2823cfb4273a94909f2c485ac95'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:04:07.807343 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-6801c2e9-872e-4f85-a0cb-d7c77a631f93 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '82dfb2823cfb4273a94909f2c485ac95'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:04:07.808122 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-6801c2e9-872e-4f85-a0cb-d7c77a631f93 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '82dfb2823cfb4273a94909f2c485ac95'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:04:07.860582 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-6801c2e9-872e-4f85-a0cb-d7c77a631f93 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '82dfb2823cfb4273a94909f2c485ac95'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:04:07.883482 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=88,keystone:INSERT=3,keystone:DELETE=9 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:04:07.886226 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-6801c2e9-872e-4f85-a0cb-d7c77a631f93 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '82dfb2823cfb4273a94909f2c485ac95'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:04:07.892450 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 740/1507] 10.222.0.46 () {58 vars in 1224 bytes} [Mon Oct 10 12:04:07 2022] DELETE /identity/v3/users/82dfb2823cfb4273a94909f2c485ac95 => generated 0 bytes in 419 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:04:07.897158 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c6c86b-69dc-490f-b8c1-544a1807a8e4 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:04:07.897443 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c6c86b-69dc-490f-b8c1-544a1807a8e4 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:04:07.897718 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c6c86b-69dc-490f-b8c1-544a1807a8e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:04:07.917136 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-e8c6c86b-69dc-490f-b8c1-544a1807a8e4 None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:04:07.918393 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-e8c6c86b-69dc-490f-b8c1-544a1807a8e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:04:07.928250 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:04:07.978915 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 768/1508] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:04:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:04:07.988970 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6801c2e9-872e-4f85-a0cb-d7c77a631f93 admin admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:04:07.993736 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-6801c2e9-872e-4f85-a0cb-d7c77a631f93 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:04:08.073312 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6801c2e9-872e-4f85-a0cb-d7c77a631f93 admin admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:04:08.076953 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab5d77f-6ae1-455c-af6d-f2a9dc57fc57 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58b1c7c85302421cba823ace4201861f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad3772f7b6a47c28b7ba37a9316a3c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:04:08.077575 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab5d77f-6ae1-455c-af6d-f2a9dc57fc57 service neutron] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:04:08.077854 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab5d77f-6ae1-455c-af6d-f2a9dc57fc57 service neutron] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:04:08.078130 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab5d77f-6ae1-455c-af6d-f2a9dc57fc57 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:04:08.087536 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab5d77f-6ae1-455c-af6d-f2a9dc57fc57 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:04:08.088981 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab5d77f-6ae1-455c-af6d-f2a9dc57fc57 service neutron] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:04:08.099436 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 741/1509] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 10 12:04:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:04:08.587079 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c6c86b-69dc-490f-b8c1-544a1807a8e4 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:04:08.588663 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-e8c6c86b-69dc-490f-b8c1-544a1807a8e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:04:08.601835 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c6c86b-69dc-490f-b8c1-544a1807a8e4 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:04:08.606830 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dff46c8-2bbd-4a0f-9081-bdb29671ed12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:04:08.607495 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dff46c8-2bbd-4a0f-9081-bdb29671ed12 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:04:08.607778 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dff46c8-2bbd-4a0f-9081-bdb29671ed12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:04:08.608055 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dff46c8-2bbd-4a0f-9081-bdb29671ed12 admin admin] PATH_INFO: `/v3/projects/b093cd038d8a44499dfffe2333d0a537` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:04:08.610069 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dff46c8-2bbd-4a0f-9081-bdb29671ed12 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b093cd038d8a44499dfffe2333d0a537)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:04:08.610539 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: WARNING py.warnings [None req-5dff46c8-2bbd-4a0f-9081-bdb29671ed12 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 10 12:04:08.610539 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: warnings.warn(msg) Oct 10 12:04:08.610539 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: Oct 10 12:04:08.611835 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dff46c8-2bbd-4a0f-9081-bdb29671ed12 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:04:08.723503 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-5dff46c8-2bbd-4a0f-9081-bdb29671ed12 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b093cd038d8a44499dfffe2333d0a537', 'request_id': 'req-5dff46c8-2bbd-4a0f-9081-bdb29671ed12'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:04:08.729658 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-5dff46c8-2bbd-4a0f-9081-bdb29671ed12 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b093cd038d8a44499dfffe2333d0a537', 'request_id': 'req-5dff46c8-2bbd-4a0f-9081-bdb29671ed12'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:04:08.743375 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-5dff46c8-2bbd-4a0f-9081-bdb29671ed12 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b093cd038d8a44499dfffe2333d0a537', 'request_id': 'req-5dff46c8-2bbd-4a0f-9081-bdb29671ed12'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:04:08.807566 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-5dff46c8-2bbd-4a0f-9081-bdb29671ed12 admin admin] The token cache is being invalidate because project b093cd038d8a44499dfffe2333d0a537 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81677) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 10 12:04:08.807860 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-5dff46c8-2bbd-4a0f-9081-bdb29671ed12 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:04:08.808507 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-5dff46c8-2bbd-4a0f-9081-bdb29671ed12 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:04:08.810170 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 769/1510] 10.222.0.46 () {58 vars in 1233 bytes} [Mon Oct 10 12:04:08 2022] DELETE /identity/v3/projects/b093cd038d8a44499dfffe2333d0a537 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:04:18.094380 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:DELETE=1,keystone:SELECT=37 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:04:18.804237 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=47,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:05:39.012774 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1446b38e-7f33-4590-bd89-11578e70bcef None None] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:05:39.013384 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1446b38e-7f33-4590-bd89-11578e70bcef None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:05:39.013384 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1446b38e-7f33-4590-bd89-11578e70bcef None None] PATH_INFO: `/` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:05:39.014724 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 742/1511] 127.0.0.1 () {54 vars in 901 bytes} [Mon Oct 10 12:05:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 12:05:39.019818 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e70d8dd-e6d8-4a0b-a9da-3980b021a69e None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:05:39.020266 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e70d8dd-e6d8-4a0b-a9da-3980b021a69e None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:05:39.020370 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e70d8dd-e6d8-4a0b-a9da-3980b021a69e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:05:39.040394 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-1e70d8dd-e6d8-4a0b-a9da-3980b021a69e None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:05:39.041581 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-1e70d8dd-e6d8-4a0b-a9da-3980b021a69e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:05:39.100978 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 770/1512] 127.0.0.1 () {58 vars in 998 bytes} [Mon Oct 10 12:05:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 10 12:05:39.255441 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1446b38e-7f33-4590-bd89-11578e70bcef None None] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:05:39.257215 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-1446b38e-7f33-4590-bd89-11578e70bcef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:05:39.361322 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1446b38e-7f33-4590-bd89-11578e70bcef None None] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:05:39.366818 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a59a544-903b-4fd6-9206-f8a717b462d1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f130bbb69e546ce99b4e14de81d7e54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad3772f7b6a47c28b7ba37a9316a3c1', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:05:39.367423 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a59a544-903b-4fd6-9206-f8a717b462d1 service nova] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:05:39.367815 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a59a544-903b-4fd6-9206-f8a717b462d1 service nova] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:05:39.368578 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a59a544-903b-4fd6-9206-f8a717b462d1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:05:39.376556 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a59a544-903b-4fd6-9206-f8a717b462d1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:05:39.378133 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a59a544-903b-4fd6-9206-f8a717b462d1 service nova] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:05:39.387348 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 743/1513] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 10 12:05:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 10 12:05:40.472940 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910766c6-d32d-4c29-b0d5-4295957c3a89 None None] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:05:40.473488 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910766c6-d32d-4c29-b0d5-4295957c3a89 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:05:40.473488 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910766c6-d32d-4c29-b0d5-4295957c3a89 None None] PATH_INFO: `/` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:05:40.474859 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 771/1514] 127.0.0.1 () {54 vars in 901 bytes} [Mon Oct 10 12:05:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 12:05:40.482137 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b22658-5ee7-4e4b-85ba-d8a4c886ca49 None None] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:05:40.482644 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b22658-5ee7-4e4b-85ba-d8a4c886ca49 None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:05:40.482789 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b22658-5ee7-4e4b-85ba-d8a4c886ca49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:05:40.502722 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.auth.core [None req-59b22658-5ee7-4e4b-85ba-d8a4c886ca49 None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:05:40.503930 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-59b22658-5ee7-4e4b-85ba-d8a4c886ca49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:05:40.512397 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 744/1515] 127.0.0.1 () {58 vars in 998 bytes} [Mon Oct 10 12:05:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:05:40.615392 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6975685a-ac1b-4c4b-a729-a8cda9777bd0 None None] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:05:40.615683 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6975685a-ac1b-4c4b-a729-a8cda9777bd0 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:05:40.615965 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6975685a-ac1b-4c4b-a729-a8cda9777bd0 None None] PATH_INFO: `/v3` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:05:40.617245 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 772/1516] 10.222.0.48 () {54 vars in 925 bytes} [Mon Oct 10 12:05:40 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 12:05:40.623780 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b913301-dc8a-4f29-a927-84d1d392f2fa None None] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:05:40.624067 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b913301-dc8a-4f29-a927-84d1d392f2fa None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:05:40.624345 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b913301-dc8a-4f29-a927-84d1d392f2fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:05:40.666617 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6975685a-ac1b-4c4b-a729-a8cda9777bd0 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:05:40.676124 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.auth.core [None req-0b913301-dc8a-4f29-a927-84d1d392f2fa None None] MFA Rules not processed for user `8e8f9bb1776d47169ef7febb57c954cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:05:40.677341 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-0b913301-dc8a-4f29-a927-84d1d392f2fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:05:40.681132 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6975685a-ac1b-4c4b-a729-a8cda9777bd0 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:05:40.686609 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319e19b1-ab8c-4915-b726-efd171f6037d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f130bbb69e546ce99b4e14de81d7e54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad3772f7b6a47c28b7ba37a9316a3c1', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:05:40.687247 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319e19b1-ab8c-4915-b726-efd171f6037d service nova] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:05:40.687532 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319e19b1-ab8c-4915-b726-efd171f6037d service nova] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:05:40.687811 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319e19b1-ab8c-4915-b726-efd171f6037d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:05:40.696130 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319e19b1-ab8c-4915-b726-efd171f6037d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:05:40.697709 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319e19b1-ab8c-4915-b726-efd171f6037d service nova] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:05:40.707540 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 773/1517] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 10 12:05:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:05:40.751898 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 745/1518] 10.222.0.48 () {58 vars in 1015 bytes} [Mon Oct 10 12:05:40 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 10 12:05:49.949739 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e76d87f-aedc-4adc-9750-4bc745d2b1eb None None] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:05:49.950223 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e76d87f-aedc-4adc-9750-4bc745d2b1eb None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:05:49.950432 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e76d87f-aedc-4adc-9750-4bc745d2b1eb None None] PATH_INFO: `/` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:05:49.953149 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 774/1519] 10.222.0.46 () {54 vars in 961 bytes} [Mon Oct 10 12:05:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 12:05:49.959411 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b69bddd-30d1-4159-8030-a1d8d1f3af2e None None] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:05:49.962630 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b69bddd-30d1-4159-8030-a1d8d1f3af2e None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:05:49.962630 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b69bddd-30d1-4159-8030-a1d8d1f3af2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:05:49.980493 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.auth.core [None req-6b69bddd-30d1-4159-8030-a1d8d1f3af2e None None] MFA Rules not processed for user `8e8f9bb1776d47169ef7febb57c954cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:05:49.982025 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-6b69bddd-30d1-4159-8030-a1d8d1f3af2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:05:49.991442 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 746/1520] 10.222.0.46 () {58 vars in 1058 bytes} [Mon Oct 10 12:05:49 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:05:49.999816 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e76d87f-aedc-4adc-9750-4bc745d2b1eb None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:05:50.015585 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e76d87f-aedc-4adc-9750-4bc745d2b1eb None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:05:50.020887 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f2ff57-6078-4fcd-890c-02f5ee8c1a61 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e8f9bb1776d47169ef7febb57c954cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad3772f7b6a47c28b7ba37a9316a3c1', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:05:50.021505 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f2ff57-6078-4fcd-890c-02f5ee8c1a61 service placement] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:05:50.021790 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f2ff57-6078-4fcd-890c-02f5ee8c1a61 service placement] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:05:50.022071 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f2ff57-6078-4fcd-890c-02f5ee8c1a61 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:05:50.031087 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f2ff57-6078-4fcd-890c-02f5ee8c1a61 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:05:50.032664 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f2ff57-6078-4fcd-890c-02f5ee8c1a61 service placement] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:05:50.042478 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 775/1521] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 10 12:05:49 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:05:52.155859 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fd6d33-32b9-437c-946f-7a8c1791b93b None None] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:05:52.156442 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fd6d33-32b9-437c-946f-7a8c1791b93b None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:05:52.156442 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fd6d33-32b9-437c-946f-7a8c1791b93b None None] PATH_INFO: `/` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:05:52.157792 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 747/1522] 127.0.0.1 () {54 vars in 901 bytes} [Mon Oct 10 12:05:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 12:05:52.164064 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f2caff-ee85-4220-be91-f3649313b932 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:05:52.164495 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f2caff-ee85-4220-be91-f3649313b932 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:05:52.164592 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f2caff-ee85-4220-be91-f3649313b932 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:05:52.185003 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-e5f2caff-ee85-4220-be91-f3649313b932 None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:05:52.186194 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-e5f2caff-ee85-4220-be91-f3649313b932 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:05:52.193938 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 776/1523] 127.0.0.1 () {58 vars in 998 bytes} [Mon Oct 10 12:05:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:05:52.350424 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86fd6d33-32b9-437c-946f-7a8c1791b93b None None] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:05:52.365711 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86fd6d33-32b9-437c-946f-7a8c1791b93b None None] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:05:52.371170 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0872a000-0be7-4ee2-9dac-d8013ea5967e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f130bbb69e546ce99b4e14de81d7e54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad3772f7b6a47c28b7ba37a9316a3c1', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:05:52.371792 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0872a000-0be7-4ee2-9dac-d8013ea5967e service nova] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:05:52.372054 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0872a000-0be7-4ee2-9dac-d8013ea5967e service nova] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:05:52.372327 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0872a000-0be7-4ee2-9dac-d8013ea5967e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:05:52.378348 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:05:52.380552 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0872a000-0be7-4ee2-9dac-d8013ea5967e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:05:52.382118 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0872a000-0be7-4ee2-9dac-d8013ea5967e service nova] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:05:52.395013 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 748/1524] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 10 12:05:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:05:53.467101 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68780d1-bbb0-4029-a65d-3d87b7b2a149 None None] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:05:53.467666 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68780d1-bbb0-4029-a65d-3d87b7b2a149 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:05:53.467666 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68780d1-bbb0-4029-a65d-3d87b7b2a149 None None] PATH_INFO: `/` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:05:53.468996 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 777/1525] 127.0.0.1 () {54 vars in 901 bytes} [Mon Oct 10 12:05:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 12:05:53.475438 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bed99fc-3e65-45cd-a967-9acfc8a82511 None None] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:05:53.475881 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bed99fc-3e65-45cd-a967-9acfc8a82511 None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:05:53.476019 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bed99fc-3e65-45cd-a967-9acfc8a82511 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:05:53.496692 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.auth.core [None req-7bed99fc-3e65-45cd-a967-9acfc8a82511 None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:05:53.497892 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-7bed99fc-3e65-45cd-a967-9acfc8a82511 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:05:53.505563 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 749/1526] 127.0.0.1 () {58 vars in 998 bytes} [Mon Oct 10 12:05:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 10 12:05:53.656715 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68780d1-bbb0-4029-a65d-3d87b7b2a149 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:05:53.671498 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68780d1-bbb0-4029-a65d-3d87b7b2a149 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:05:53.676965 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afa2ff9-864f-42f9-9f0b-728ae1db2e7d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f130bbb69e546ce99b4e14de81d7e54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad3772f7b6a47c28b7ba37a9316a3c1', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:05:53.677563 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afa2ff9-864f-42f9-9f0b-728ae1db2e7d service nova] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:05:53.677848 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afa2ff9-864f-42f9-9f0b-728ae1db2e7d service nova] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:05:53.678126 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afa2ff9-864f-42f9-9f0b-728ae1db2e7d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:05:53.686847 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1afa2ff9-864f-42f9-9f0b-728ae1db2e7d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:05:53.688418 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1afa2ff9-864f-42f9-9f0b-728ae1db2e7d service nova] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:05:53.697745 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 778/1527] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 10 12:05:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:05:58.117917 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9d63a5-f35f-405e-b812-564888fc50b7 None None] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:05:58.118493 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9d63a5-f35f-405e-b812-564888fc50b7 None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:05:58.118493 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9d63a5-f35f-405e-b812-564888fc50b7 None None] PATH_INFO: `/` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:05:58.119845 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 750/1528] 127.0.0.1 () {54 vars in 917 bytes} [Mon Oct 10 12:05:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 12:05:58.125117 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f45548f-fea1-4bae-91f3-d3082c1a16a9 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:05:58.125517 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f45548f-fea1-4bae-91f3-d3082c1a16a9 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:05:58.125671 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f45548f-fea1-4bae-91f3-d3082c1a16a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:05:58.145423 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-7f45548f-fea1-4bae-91f3-d3082c1a16a9 None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:05:58.146619 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-7f45548f-fea1-4bae-91f3-d3082c1a16a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:05:58.155088 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 779/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Oct 10 12:05:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:05:58.444024 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9d63a5-f35f-405e-b812-564888fc50b7 None None] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:05:58.445808 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-7a9d63a5-f35f-405e-b812-564888fc50b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:05:58.504801 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9d63a5-f35f-405e-b812-564888fc50b7 None None] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:05:58.509112 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de99cb95-282b-4785-8c5a-ba9a2303a9e6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58b1c7c85302421cba823ace4201861f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad3772f7b6a47c28b7ba37a9316a3c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:05:58.509717 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de99cb95-282b-4785-8c5a-ba9a2303a9e6 service neutron] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:05:58.510001 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de99cb95-282b-4785-8c5a-ba9a2303a9e6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:05:58.510280 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de99cb95-282b-4785-8c5a-ba9a2303a9e6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:05:58.519229 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de99cb95-282b-4785-8c5a-ba9a2303a9e6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:05:58.520654 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de99cb95-282b-4785-8c5a-ba9a2303a9e6 service neutron] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:05:58.529954 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 751/1530] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 10 12:05:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:06:00.683078 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851ea6fe-d9de-4ea6-ac88-c13f7db5594b None None] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:06:00.683608 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851ea6fe-d9de-4ea6-ac88-c13f7db5594b None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:06:00.683608 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851ea6fe-d9de-4ea6-ac88-c13f7db5594b None None] PATH_INFO: `/` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:06:00.685431 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 780/1531] 127.0.0.1 () {54 vars in 917 bytes} [Mon Oct 10 12:06:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 12:06:00.690801 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d908eb-0352-41c2-8232-9e74475c17b7 None None] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:06:00.691225 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d908eb-0352-41c2-8232-9e74475c17b7 None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:06:00.691396 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d908eb-0352-41c2-8232-9e74475c17b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:06:00.711176 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.auth.core [None req-63d908eb-0352-41c2-8232-9e74475c17b7 None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:06:00.714137 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-63d908eb-0352-41c2-8232-9e74475c17b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:06:00.720909 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 752/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Oct 10 12:06:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:06:01.018466 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851ea6fe-d9de-4ea6-ac88-c13f7db5594b None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:06:01.020210 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-851ea6fe-d9de-4ea6-ac88-c13f7db5594b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:06:01.033648 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851ea6fe-d9de-4ea6-ac88-c13f7db5594b None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:06:01.037168 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd0c4fa-c26d-49be-bddb-2be004d8c0ce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58b1c7c85302421cba823ace4201861f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad3772f7b6a47c28b7ba37a9316a3c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:06:01.037766 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd0c4fa-c26d-49be-bddb-2be004d8c0ce service neutron] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:06:01.038049 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd0c4fa-c26d-49be-bddb-2be004d8c0ce service neutron] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:06:01.038332 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd0c4fa-c26d-49be-bddb-2be004d8c0ce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:06:01.047274 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd0c4fa-c26d-49be-bddb-2be004d8c0ce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:06:01.048681 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd0c4fa-c26d-49be-bddb-2be004d8c0ce service neutron] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:06:01.057881 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 781/1533] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 10 12:06:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:06:04.697578 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20c0539-93ba-47f5-ad87-d2691bd186b0 None None] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:06:04.698138 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20c0539-93ba-47f5-ad87-d2691bd186b0 None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:06:04.698138 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20c0539-93ba-47f5-ad87-d2691bd186b0 None None] PATH_INFO: `/` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:06:04.699440 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 753/1534] 127.0.0.1 () {54 vars in 917 bytes} [Mon Oct 10 12:06:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 12:06:04.704934 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4daef6b-b425-4b43-a17f-890c4c632fea None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:06:04.705412 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4daef6b-b425-4b43-a17f-890c4c632fea None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:06:04.705559 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4daef6b-b425-4b43-a17f-890c4c632fea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:06:04.724810 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-b4daef6b-b425-4b43-a17f-890c4c632fea None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:06:04.726007 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-b4daef6b-b425-4b43-a17f-890c4c632fea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:06:04.734774 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 782/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Oct 10 12:06:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:06:05.040821 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20c0539-93ba-47f5-ad87-d2691bd186b0 None None] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:06:05.042516 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-b20c0539-93ba-47f5-ad87-d2691bd186b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:06:05.098851 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20c0539-93ba-47f5-ad87-d2691bd186b0 None None] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:06:05.103184 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef48b915-5045-442e-9837-e4917080e837 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3ae093470484b93b23dcfae1657df9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad3772f7b6a47c28b7ba37a9316a3c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:06:05.103792 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef48b915-5045-442e-9837-e4917080e837 service cinder] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:06:05.104094 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef48b915-5045-442e-9837-e4917080e837 service cinder] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:06:05.104338 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef48b915-5045-442e-9837-e4917080e837 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:06:05.114483 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef48b915-5045-442e-9837-e4917080e837 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:06:05.115970 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef48b915-5045-442e-9837-e4917080e837 service cinder] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:06:05.125629 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 754/1536] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 10 12:06:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:06:07.266068 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239cbefe-2668-4a02-bb07-600845b6f146 None None] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:06:07.266645 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239cbefe-2668-4a02-bb07-600845b6f146 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:06:07.266645 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239cbefe-2668-4a02-bb07-600845b6f146 None None] PATH_INFO: `/` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:06:07.267995 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 783/1537] 127.0.0.1 () {54 vars in 917 bytes} [Mon Oct 10 12:06:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 12:06:07.273592 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4bcd98-886b-4884-97d5-5cb7c3cadc4f None None] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:06:07.274079 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4bcd98-886b-4884-97d5-5cb7c3cadc4f None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:06:07.274195 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4bcd98-886b-4884-97d5-5cb7c3cadc4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:06:07.293702 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.auth.core [None req-de4bcd98-886b-4884-97d5-5cb7c3cadc4f None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:06:07.294965 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-de4bcd98-886b-4884-97d5-5cb7c3cadc4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:06:07.303049 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 755/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Oct 10 12:06:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:06:07.605239 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239cbefe-2668-4a02-bb07-600845b6f146 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:06:07.607244 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-239cbefe-2668-4a02-bb07-600845b6f146 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:06:07.620843 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239cbefe-2668-4a02-bb07-600845b6f146 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:06:07.624389 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a49b265-1e6e-4fe8-82e4-edc9a3d5019a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3ae093470484b93b23dcfae1657df9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad3772f7b6a47c28b7ba37a9316a3c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:06:07.624981 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a49b265-1e6e-4fe8-82e4-edc9a3d5019a service cinder] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:06:07.625265 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a49b265-1e6e-4fe8-82e4-edc9a3d5019a service cinder] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:06:07.625542 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a49b265-1e6e-4fe8-82e4-edc9a3d5019a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:06:07.634239 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a49b265-1e6e-4fe8-82e4-edc9a3d5019a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:06:07.635693 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a49b265-1e6e-4fe8-82e4-edc9a3d5019a service cinder] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:06:07.644880 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 784/1539] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 10 12:06:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:06:17.288789 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=63 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:06:17.640766 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=66 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:07:33.955039 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6628f59d-3423-447a-96a3-12503398a0f9 None None] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:07:33.955684 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6628f59d-3423-447a-96a3-12503398a0f9 None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:07:33.955684 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6628f59d-3423-447a-96a3-12503398a0f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:07:33.980939 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.auth.core [None req-6628f59d-3423-447a-96a3-12503398a0f9 None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:07:33.982559 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-6628f59d-3423-447a-96a3-12503398a0f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:07:33.995899 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 756/1540] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:07:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:07:34.005252 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd97390f-4aa0-4483-8613-165cc346f20f None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:07:34.005841 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd97390f-4aa0-4483-8613-165cc346f20f None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:07:34.005841 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd97390f-4aa0-4483-8613-165cc346f20f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:07:34.040044 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-cd97390f-4aa0-4483-8613-165cc346f20f None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:07:34.041499 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-cd97390f-4aa0-4483-8613-165cc346f20f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:07:34.050830 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 785/1541] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:07:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:07:34.055660 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6628f59d-3423-447a-96a3-12503398a0f9 None None] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:07:34.073942 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6628f59d-3423-447a-96a3-12503398a0f9 None None] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:07:34.086014 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24afdfc-7c9a-4169-9d10-f3edec2683ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:07:34.087755 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24afdfc-7c9a-4169-9d10-f3edec2683ab admin admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:07:34.088365 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24afdfc-7c9a-4169-9d10-f3edec2683ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:07:34.088870 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24afdfc-7c9a-4169-9d10-f3edec2683ab admin admin] PATH_INFO: `/v3/domains` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:07:34.089799 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24afdfc-7c9a-4169-9d10-f3edec2683ab admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 12:07:34.090327 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24afdfc-7c9a-4169-9d10-f3edec2683ab admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:07:34.091608 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: WARNING py.warnings [None req-f24afdfc-7c9a-4169-9d10-f3edec2683ab admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 10 12:07:34.091608 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: warnings.warn(msg) Oct 10 12:07:34.091608 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: Oct 10 12:07:34.093071 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24afdfc-7c9a-4169-9d10-f3edec2683ab admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:07:34.111552 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 757/1542] 10.222.0.46 () {58 vars in 1153 bytes} [Mon Oct 10 12:07:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 12:07:34.119987 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd97390f-4aa0-4483-8613-165cc346f20f None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:07:34.129917 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd97390f-4aa0-4483-8613-165cc346f20f None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:07:34.143007 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211bd51e-d5fc-415b-a685-0402b3174faf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:07:34.143647 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211bd51e-d5fc-415b-a685-0402b3174faf admin admin] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:07:34.143930 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211bd51e-d5fc-415b-a685-0402b3174faf admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:07:34.144208 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211bd51e-d5fc-415b-a685-0402b3174faf admin admin] PATH_INFO: `/v3/projects` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:07:34.144852 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211bd51e-d5fc-415b-a685-0402b3174faf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:07:34.146165 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: WARNING py.warnings [None req-211bd51e-d5fc-415b-a685-0402b3174faf admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 10 12:07:34.146165 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: warnings.warn(msg) Oct 10 12:07:34.146165 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: Oct 10 12:07:34.147394 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211bd51e-d5fc-415b-a685-0402b3174faf admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:07:34.205190 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aed0b02-ba35-4122-9d34-d0842b69785d None None] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:07:34.205868 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aed0b02-ba35-4122-9d34-d0842b69785d None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:07:34.206472 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aed0b02-ba35-4122-9d34-d0842b69785d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:07:34.241864 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.auth.core [None req-8aed0b02-ba35-4122-9d34-d0842b69785d None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:07:34.244110 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-8aed0b02-ba35-4122-9d34-d0842b69785d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:07:34.257272 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 758/1543] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:07:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 10 12:07:34.267988 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b84fe9-ddf7-4c55-83b4-9548483aa15b None None] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:07:34.268864 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b84fe9-ddf7-4c55-83b4-9548483aa15b None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:07:34.269356 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b84fe9-ddf7-4c55-83b4-9548483aa15b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:07:34.289406 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 786/1544] 10.222.0.46 () {60 vars in 1153 bytes} [Mon Oct 10 12:07:34 2022] POST /identity/v3/projects => generated 373 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 12:07:34.295730 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211bd51e-d5fc-415b-a685-0402b3174faf admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:07:34.301899 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.auth.core [None req-d1b84fe9-ddf7-4c55-83b4-9548483aa15b None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:07:34.303699 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-d1b84fe9-ddf7-4c55-83b4-9548483aa15b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:07:34.407590 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211bd51e-d5fc-415b-a685-0402b3174faf admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:07:34.414241 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 759/1545] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:07:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 10 12:07:34.415852 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8931fd6-fbf7-4e14-8370-e3eb80750762 None None] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:07:34.416360 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8931fd6-fbf7-4e14-8370-e3eb80750762 None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:07:34.416961 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8931fd6-fbf7-4e14-8370-e3eb80750762 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:07:34.417316 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e482cd-44a2-408b-8c40-b01eccaf68fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:07:34.417959 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e482cd-44a2-408b-8c40-b01eccaf68fb admin admin] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:07:34.418248 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e482cd-44a2-408b-8c40-b01eccaf68fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:07:34.418525 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e482cd-44a2-408b-8c40-b01eccaf68fb admin admin] PATH_INFO: `/v3/users` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:07:34.419205 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e482cd-44a2-408b-8c40-b01eccaf68fb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:07:34.421199 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e482cd-44a2-408b-8c40-b01eccaf68fb admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:07:34.473233 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.auth.core [None req-f8931fd6-fbf7-4e14-8370-e3eb80750762 None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:07:34.473233 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-f8931fd6-fbf7-4e14-8370-e3eb80750762 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:07:34.501614 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 760/1546] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:07:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:07:34.502320 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8931fd6-fbf7-4e14-8370-e3eb80750762 None None] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:07:34.522400 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8931fd6-fbf7-4e14-8370-e3eb80750762 None None] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:07:34.530890 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 787/1547] 10.222.0.46 () {60 vars in 1144 bytes} [Mon Oct 10 12:07:34 2022] POST /identity/v3/users => generated 287 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 12:07:34.532626 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c693c5-0a31-4869-9052-a360843f4c79 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:07:34.533656 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c693c5-0a31-4869-9052-a360843f4c79 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:07:34.534473 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd61066d-347e-4323-b5a7-8ce116b6360f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:07:34.535128 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd61066d-347e-4323-b5a7-8ce116b6360f admin admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:07:34.535407 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd61066d-347e-4323-b5a7-8ce116b6360f admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:07:34.535683 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd61066d-347e-4323-b5a7-8ce116b6360f admin admin] PATH_INFO: `/v3/domains` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:07:34.536348 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd61066d-347e-4323-b5a7-8ce116b6360f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 12:07:34.536652 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd61066d-347e-4323-b5a7-8ce116b6360f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:07:34.538269 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c693c5-0a31-4869-9052-a360843f4c79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:07:34.542440 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd61066d-347e-4323-b5a7-8ce116b6360f admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:07:34.562025 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 761/1548] 10.222.0.46 () {58 vars in 1153 bytes} [Mon Oct 10 12:07:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 12:07:34.563845 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b0d593-64ef-445b-98df-d44173655ac2 None None] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:07:34.564379 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b0d593-64ef-445b-98df-d44173655ac2 None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:07:34.564863 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b0d593-64ef-445b-98df-d44173655ac2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:07:34.576609 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-65c693c5-0a31-4869-9052-a360843f4c79 None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:07:34.578368 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-65c693c5-0a31-4869-9052-a360843f4c79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:07:34.589544 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.auth.core [None req-55b0d593-64ef-445b-98df-d44173655ac2 None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:07:34.591002 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 788/1549] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:07:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:07:34.591139 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-55b0d593-64ef-445b-98df-d44173655ac2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:07:34.594810 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6670e442-c004-48fd-a6b7-5f5213873421 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:07:34.594810 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6670e442-c004-48fd-a6b7-5f5213873421 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:07:34.595136 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6670e442-c004-48fd-a6b7-5f5213873421 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:07:34.610117 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 762/1550] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:07:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:07:34.611630 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b0d593-64ef-445b-98df-d44173655ac2 None None] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:07:34.620490 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-6670e442-c004-48fd-a6b7-5f5213873421 None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:07:34.621888 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-6670e442-c004-48fd-a6b7-5f5213873421 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:07:34.622163 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b0d593-64ef-445b-98df-d44173655ac2 None None] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:07:34.627252 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e0595f-b4fa-4b99-9473-0a5385f96e2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:07:34.627857 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e0595f-b4fa-4b99-9473-0a5385f96e2e admin admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:07:34.628210 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e0595f-b4fa-4b99-9473-0a5385f96e2e admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:07:34.628421 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e0595f-b4fa-4b99-9473-0a5385f96e2e admin admin] PATH_INFO: `/v3/roles` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:07:34.629213 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e0595f-b4fa-4b99-9473-0a5385f96e2e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:07:34.630311 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 789/1551] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:07:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:07:34.631682 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e0595f-b4fa-4b99-9473-0a5385f96e2e admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:07:34.634755 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19aecc22-2930-4a43-8d3d-dbfb4ce1de8a None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:07:34.634755 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19aecc22-2930-4a43-8d3d-dbfb4ce1de8a None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:07:34.634755 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19aecc22-2930-4a43-8d3d-dbfb4ce1de8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:07:34.642728 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 763/1552] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 10 12:07:34 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 12:07:34.643695 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e0595f-b4fa-4b99-9473-0a5385f96e2e admin admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:07:34.655519 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e0595f-b4fa-4b99-9473-0a5385f96e2e admin admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:07:34.661206 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-19aecc22-2930-4a43-8d3d-dbfb4ce1de8a None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:07:34.661206 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-19aecc22-2930-4a43-8d3d-dbfb4ce1de8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:07:34.661462 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9ecca3-5f18-4304-a7f7-75be194efdc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:07:34.661462 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9ecca3-5f18-4304-a7f7-75be194efdc3 admin admin] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:07:34.661462 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9ecca3-5f18-4304-a7f7-75be194efdc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:07:34.661694 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9ecca3-5f18-4304-a7f7-75be194efdc3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:07:34.662286 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9ecca3-5f18-4304-a7f7-75be194efdc3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:07:34.665397 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9ecca3-5f18-4304-a7f7-75be194efdc3 admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:07:34.669365 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 790/1553] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:07:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:07:34.670699 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a273ae-9da5-4299-bf29-06e97195e484 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:07:34.671080 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a273ae-9da5-4299-bf29-06e97195e484 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:07:34.671375 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a273ae-9da5-4299-bf29-06e97195e484 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:07:34.691919 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-60a273ae-9da5-4299-bf29-06e97195e484 None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:07:34.697728 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-60a273ae-9da5-4299-bf29-06e97195e484 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:07:34.701923 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 791/1554] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:07:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:07:34.703204 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db679c23-88cb-40c0-975e-27ed7cfa326b None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:07:34.703489 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db679c23-88cb-40c0-975e-27ed7cfa326b None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:07:34.703769 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db679c23-88cb-40c0-975e-27ed7cfa326b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:07:34.726381 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-db679c23-88cb-40c0-975e-27ed7cfa326b None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:07:34.726381 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-db679c23-88cb-40c0-975e-27ed7cfa326b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:07:34.734018 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 792/1555] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:07:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:07:34.734863 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db679c23-88cb-40c0-975e-27ed7cfa326b None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:07:34.745353 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db679c23-88cb-40c0-975e-27ed7cfa326b None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:07:34.750883 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd81c50-c78d-4bda-a649-5de073f5c859 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:07:34.750883 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd81c50-c78d-4bda-a649-5de073f5c859 admin admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:07:34.751121 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd81c50-c78d-4bda-a649-5de073f5c859 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:07:34.751269 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd81c50-c78d-4bda-a649-5de073f5c859 admin admin] PATH_INFO: `/v3/domains` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:07:34.751947 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd81c50-c78d-4bda-a649-5de073f5c859 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 12:07:34.752252 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd81c50-c78d-4bda-a649-5de073f5c859 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:07:34.753850 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd81c50-c78d-4bda-a649-5de073f5c859 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:07:34.766024 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 793/1556] 10.222.0.46 () {58 vars in 1153 bytes} [Mon Oct 10 12:07:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 12:07:34.767280 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0156053-074c-4666-b8f8-8e02397c2774 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:07:34.767624 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0156053-074c-4666-b8f8-8e02397c2774 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:07:34.767954 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0156053-074c-4666-b8f8-8e02397c2774 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:07:34.787902 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-b0156053-074c-4666-b8f8-8e02397c2774 None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:07:34.789624 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-b0156053-074c-4666-b8f8-8e02397c2774 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:07:34.797229 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 764/1557] 10.222.0.46 () {60 vars in 1153 bytes} [Mon Oct 10 12:07:34 2022] POST /identity/v3/projects => generated 385 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 12:07:34.797229 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9ecca3-5f18-4304-a7f7-75be194efdc3 admin admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:07:34.797806 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 794/1558] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:07:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:07:34.799062 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174cadff-3f6d-4bd1-b4f6-5cbd197156a9 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:07:34.799348 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174cadff-3f6d-4bd1-b4f6-5cbd197156a9 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:07:34.799634 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174cadff-3f6d-4bd1-b4f6-5cbd197156a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:07:34.821152 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-174cadff-3f6d-4bd1-b4f6-5cbd197156a9 None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:07:34.822483 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-174cadff-3f6d-4bd1-b4f6-5cbd197156a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:07:34.864676 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9ecca3-5f18-4304-a7f7-75be194efdc3 admin admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:07:34.870019 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693470f6-1f03-4544-b7e0-0d9d002a860b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:07:34.870634 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693470f6-1f03-4544-b7e0-0d9d002a860b admin admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:07:34.870966 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693470f6-1f03-4544-b7e0-0d9d002a860b admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:07:34.871333 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693470f6-1f03-4544-b7e0-0d9d002a860b admin admin] PATH_INFO: `/v3/roles` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:07:34.871863 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693470f6-1f03-4544-b7e0-0d9d002a860b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:07:34.873475 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693470f6-1f03-4544-b7e0-0d9d002a860b admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:07:34.883816 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 795/1559] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:07:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:07:34.884207 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 765/1560] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 10 12:07:34 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 12:07:34.885122 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d95909-12ee-4d2f-aff7-8c8be47f9080 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:07:34.885415 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d95909-12ee-4d2f-aff7-8c8be47f9080 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:07:34.885527 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced4b8bf-3047-4718-a0c2-f0341e79e3e3 None None] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:07:34.885691 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d95909-12ee-4d2f-aff7-8c8be47f9080 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:07:34.885805 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced4b8bf-3047-4718-a0c2-f0341e79e3e3 None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:07:34.886018 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced4b8bf-3047-4718-a0c2-f0341e79e3e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:07:34.907511 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.auth.core [None req-ced4b8bf-3047-4718-a0c2-f0341e79e3e3 None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:07:34.908731 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-ced4b8bf-3047-4718-a0c2-f0341e79e3e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:07:34.910750 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-39d95909-12ee-4d2f-aff7-8c8be47f9080 None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:07:34.911936 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-39d95909-12ee-4d2f-aff7-8c8be47f9080 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:07:34.917415 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 766/1561] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:07:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:07:34.919574 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced4b8bf-3047-4718-a0c2-f0341e79e3e3 None None] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:07:34.920820 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 796/1562] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:07:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:07:34.921842 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d95909-12ee-4d2f-aff7-8c8be47f9080 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:07:34.926525 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:07:34.928321 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:07:34.931978 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced4b8bf-3047-4718-a0c2-f0341e79e3e3 None None] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:07:34.932410 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d95909-12ee-4d2f-aff7-8c8be47f9080 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:07:34.937298 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5235da-fe73-4c99-b894-738a60539a67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:07:34.937455 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2193a812-f772-48ce-a1d9-7a392f09b2b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:07:34.938361 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5235da-fe73-4c99-b894-738a60539a67 admin admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:07:34.938475 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2193a812-f772-48ce-a1d9-7a392f09b2b9 admin admin] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:07:34.938576 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5235da-fe73-4c99-b894-738a60539a67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:07:34.938698 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2193a812-f772-48ce-a1d9-7a392f09b2b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:07:34.938799 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5235da-fe73-4c99-b894-738a60539a67 admin admin] PATH_INFO: `/v3/domains` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:07:34.938884 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2193a812-f772-48ce-a1d9-7a392f09b2b9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:07:34.939151 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5235da-fe73-4c99-b894-738a60539a67 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 12:07:34.939426 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2193a812-f772-48ce-a1d9-7a392f09b2b9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:07:34.939541 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5235da-fe73-4c99-b894-738a60539a67 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:07:34.941067 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5235da-fe73-4c99-b894-738a60539a67 admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:07:34.941322 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2193a812-f772-48ce-a1d9-7a392f09b2b9 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:07:34.954298 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 767/1563] 10.222.0.46 () {58 vars in 1153 bytes} [Mon Oct 10 12:07:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 12:07:34.955548 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5235da-fe73-4c99-b894-738a60539a67 admin admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:07:34.967882 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5235da-fe73-4c99-b894-738a60539a67 admin admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:07:34.972949 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6223e751-185e-45f6-a6b0-b7aa513dad68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:07:34.973676 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6223e751-185e-45f6-a6b0-b7aa513dad68 admin admin] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:07:34.973959 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6223e751-185e-45f6-a6b0-b7aa513dad68 admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:07:34.974229 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6223e751-185e-45f6-a6b0-b7aa513dad68 admin admin] PATH_INFO: `/v3/users` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:07:34.974906 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6223e751-185e-45f6-a6b0-b7aa513dad68 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:07:34.975939 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: WARNING py.warnings [None req-6223e751-185e-45f6-a6b0-b7aa513dad68 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 10 12:07:34.975939 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: warnings.warn(msg) Oct 10 12:07:34.975939 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: Oct 10 12:07:34.977894 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6223e751-185e-45f6-a6b0-b7aa513dad68 admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:07:35.046952 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 797/1564] 10.222.0.46 () {60 vars in 1153 bytes} [Mon Oct 10 12:07:34 2022] POST /identity/v3/projects => generated 375 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 12:07:35.048359 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2193a812-f772-48ce-a1d9-7a392f09b2b9 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:07:35.102218 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 768/1565] 10.222.0.46 () {60 vars in 1144 bytes} [Mon Oct 10 12:07:34 2022] POST /identity/v3/users => generated 293 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 12:07:35.103228 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6223e751-185e-45f6-a6b0-b7aa513dad68 admin admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:07:35.113792 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6223e751-185e-45f6-a6b0-b7aa513dad68 admin admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:07:35.118506 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69106904-8580-4f97-8092-a72d63dc57fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:07:35.119206 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69106904-8580-4f97-8092-a72d63dc57fb admin admin] REQUEST_METHOD: `PUT` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:07:35.119484 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69106904-8580-4f97-8092-a72d63dc57fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:07:35.119756 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69106904-8580-4f97-8092-a72d63dc57fb admin admin] PATH_INFO: `/v3/projects/b3c137fcc1464218b989ce343e84cd22/users/d30a538ba96147718bbdcb6bfa07a934/roles/b2cb2a6e818e47338ce071e6fedfc673` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:07:35.120518 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2193a812-f772-48ce-a1d9-7a392f09b2b9 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:07:35.125290 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92f2667-9ca3-4ecb-8768-cf2ff88e03f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:07:35.125908 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92f2667-9ca3-4ecb-8768-cf2ff88e03f8 admin admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:07:35.126181 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92f2667-9ca3-4ecb-8768-cf2ff88e03f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:07:35.126470 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92f2667-9ca3-4ecb-8768-cf2ff88e03f8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:07:35.127145 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92f2667-9ca3-4ecb-8768-cf2ff88e03f8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 12:07:35.132648 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92f2667-9ca3-4ecb-8768-cf2ff88e03f8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:07:35.132648 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92f2667-9ca3-4ecb-8768-cf2ff88e03f8 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:07:35.136064 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69106904-8580-4f97-8092-a72d63dc57fb admin admin] RBAC: Authorizing `identity:create_grant(project_id=b3c137fcc1464218b989ce343e84cd22, user_id=d30a538ba96147718bbdcb6bfa07a934, role_id=b2cb2a6e818e47338ce071e6fedfc673)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:07:35.138849 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69106904-8580-4f97-8092-a72d63dc57fb admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:07:35.141500 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 798/1566] 10.222.0.46 () {58 vars in 1153 bytes} [Mon Oct 10 12:07:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 12:07:35.142794 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b07ec3e-eb73-4ce8-af94-6f8f564462bc None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:07:35.143077 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b07ec3e-eb73-4ce8-af94-6f8f564462bc None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:07:35.143367 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b07ec3e-eb73-4ce8-af94-6f8f564462bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:07:35.162965 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-5b07ec3e-eb73-4ce8-af94-6f8f564462bc None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:07:35.164778 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-5b07ec3e-eb73-4ce8-af94-6f8f564462bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:07:35.172582 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 799/1567] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:07:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:07:35.173423 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b07ec3e-eb73-4ce8-af94-6f8f564462bc None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:07:35.182520 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b07ec3e-eb73-4ce8-af94-6f8f564462bc None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:07:35.187035 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654398c4-db0f-4967-b344-729d1ca7d1b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:07:35.189166 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654398c4-db0f-4967-b344-729d1ca7d1b8 admin admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:07:35.189166 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654398c4-db0f-4967-b344-729d1ca7d1b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:07:35.189166 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654398c4-db0f-4967-b344-729d1ca7d1b8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:07:35.189166 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654398c4-db0f-4967-b344-729d1ca7d1b8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 12:07:35.189166 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654398c4-db0f-4967-b344-729d1ca7d1b8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:07:35.190613 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654398c4-db0f-4967-b344-729d1ca7d1b8 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:07:35.202211 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 800/1568] 10.222.0.46 () {58 vars in 1153 bytes} [Mon Oct 10 12:07:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 12:07:35.203042 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654398c4-db0f-4967-b344-729d1ca7d1b8 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:07:35.213346 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654398c4-db0f-4967-b344-729d1ca7d1b8 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:07:35.218132 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220329c6-6fbb-48d7-a3c7-9d3b889a80f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:07:35.218724 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220329c6-6fbb-48d7-a3c7-9d3b889a80f6 admin admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:07:35.218976 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220329c6-6fbb-48d7-a3c7-9d3b889a80f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:07:35.219246 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220329c6-6fbb-48d7-a3c7-9d3b889a80f6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:07:35.219886 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220329c6-6fbb-48d7-a3c7-9d3b889a80f6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 12:07:35.220180 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220329c6-6fbb-48d7-a3c7-9d3b889a80f6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:07:35.221751 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220329c6-6fbb-48d7-a3c7-9d3b889a80f6 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:07:35.234814 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 801/1569] 10.222.0.46 () {58 vars in 1153 bytes} [Mon Oct 10 12:07:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 12:07:35.235606 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220329c6-6fbb-48d7-a3c7-9d3b889a80f6 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:07:35.245852 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220329c6-6fbb-48d7-a3c7-9d3b889a80f6 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:07:35.247974 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 769/1570] 10.222.0.46 () {60 vars in 1483 bytes} [Mon Oct 10 12:07:35 2022] PUT /identity/v3/projects/b3c137fcc1464218b989ce343e84cd22/users/d30a538ba96147718bbdcb6bfa07a934/roles/b2cb2a6e818e47338ce071e6fedfc673 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:07:35.258173 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69106904-8580-4f97-8092-a72d63dc57fb admin admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:07:35.309143 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba5e0c7-e7df-45f9-92b3-e9fae2819794 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:07:35.309755 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba5e0c7-e7df-45f9-92b3-e9fae2819794 admin admin] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:07:35.310044 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba5e0c7-e7df-45f9-92b3-e9fae2819794 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:07:35.310311 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba5e0c7-e7df-45f9-92b3-e9fae2819794 admin admin] PATH_INFO: `/v3/projects` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:07:35.311134 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba5e0c7-e7df-45f9-92b3-e9fae2819794 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:07:35.313222 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba5e0c7-e7df-45f9-92b3-e9fae2819794 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:07:35.316159 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69106904-8580-4f97-8092-a72d63dc57fb admin admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:07:35.321848 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0c25b2-3235-439d-9af1-2c0d08622e41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:07:35.322452 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0c25b2-3235-439d-9af1-2c0d08622e41 admin admin] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:07:35.327888 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0c25b2-3235-439d-9af1-2c0d08622e41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:07:35.327888 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0c25b2-3235-439d-9af1-2c0d08622e41 admin admin] PATH_INFO: `/v3/users` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:07:35.327888 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0c25b2-3235-439d-9af1-2c0d08622e41 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:07:35.327888 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0c25b2-3235-439d-9af1-2c0d08622e41 admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:07:35.408526 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 802/1571] 10.222.0.46 () {60 vars in 1153 bytes} [Mon Oct 10 12:07:35 2022] POST /identity/v3/projects => generated 375 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 12:07:35.409432 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba5e0c7-e7df-45f9-92b3-e9fae2819794 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:07:35.421083 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 770/1572] 10.222.0.46 () {60 vars in 1144 bytes} [Mon Oct 10 12:07:35 2022] POST /identity/v3/users => generated 288 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 12:07:35.421865 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0c25b2-3235-439d-9af1-2c0d08622e41 admin admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:07:35.431909 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0c25b2-3235-439d-9af1-2c0d08622e41 admin admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:07:35.442878 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:07:35.477208 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba5e0c7-e7df-45f9-92b3-e9fae2819794 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:07:35.482556 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946580ac-5b38-4780-a6fb-83dba8420139 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:07:35.483210 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946580ac-5b38-4780-a6fb-83dba8420139 admin admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:07:35.483487 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946580ac-5b38-4780-a6fb-83dba8420139 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:07:35.483759 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946580ac-5b38-4780-a6fb-83dba8420139 admin admin] PATH_INFO: `/v3/roles` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:07:35.484436 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946580ac-5b38-4780-a6fb-83dba8420139 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:07:35.486084 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946580ac-5b38-4780-a6fb-83dba8420139 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:07:35.500133 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 803/1573] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 10 12:07:35 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 12:07:35.500133 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946580ac-5b38-4780-a6fb-83dba8420139 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:07:35.504470 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ac8784-838e-462a-8708-ec1b45870e32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:07:35.505120 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ac8784-838e-462a-8708-ec1b45870e32 admin admin] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:07:35.505403 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ac8784-838e-462a-8708-ec1b45870e32 admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:07:35.505726 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ac8784-838e-462a-8708-ec1b45870e32 admin admin] PATH_INFO: `/v3/projects` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:07:35.506369 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ac8784-838e-462a-8708-ec1b45870e32 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:07:35.508958 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ac8784-838e-462a-8708-ec1b45870e32 admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:07:35.509884 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946580ac-5b38-4780-a6fb-83dba8420139 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:07:35.514784 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00f31a9-3534-48a8-9374-73a7c7ffb7c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:07:35.515915 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00f31a9-3534-48a8-9374-73a7c7ffb7c7 admin admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:07:35.516194 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00f31a9-3534-48a8-9374-73a7c7ffb7c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:07:35.516469 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00f31a9-3534-48a8-9374-73a7c7ffb7c7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:07:35.517110 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00f31a9-3534-48a8-9374-73a7c7ffb7c7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 12:07:35.517412 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00f31a9-3534-48a8-9374-73a7c7ffb7c7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:07:35.519030 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00f31a9-3534-48a8-9374-73a7c7ffb7c7 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:07:35.533266 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 804/1574] 10.222.0.46 () {58 vars in 1153 bytes} [Mon Oct 10 12:07:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 12:07:35.533266 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00f31a9-3534-48a8-9374-73a7c7ffb7c7 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:07:35.543622 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00f31a9-3534-48a8-9374-73a7c7ffb7c7 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:07:35.547516 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe196ecd-8433-4565-a76d-1a38715e9613 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:07:35.548119 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe196ecd-8433-4565-a76d-1a38715e9613 admin admin] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:07:35.548404 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe196ecd-8433-4565-a76d-1a38715e9613 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:07:35.548681 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe196ecd-8433-4565-a76d-1a38715e9613 admin admin] PATH_INFO: `/v3/projects` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:07:35.549452 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe196ecd-8433-4565-a76d-1a38715e9613 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:07:35.551521 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe196ecd-8433-4565-a76d-1a38715e9613 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:07:35.622035 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 771/1575] 10.222.0.46 () {60 vars in 1153 bytes} [Mon Oct 10 12:07:35 2022] POST /identity/v3/projects => generated 389 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 12:07:35.623110 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ac8784-838e-462a-8708-ec1b45870e32 admin admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:07:35.650831 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 805/1576] 10.222.0.46 () {60 vars in 1153 bytes} [Mon Oct 10 12:07:35 2022] POST /identity/v3/projects => generated 369 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 12:07:35.654544 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:07:35.654719 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d78513b-5ad1-480e-b2aa-8c34735737ff None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:07:35.654719 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d78513b-5ad1-480e-b2aa-8c34735737ff None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:07:35.654719 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d78513b-5ad1-480e-b2aa-8c34735737ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:07:35.681905 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-6d78513b-5ad1-480e-b2aa-8c34735737ff None None] MFA Rules not processed for user `d30a538ba96147718bbdcb6bfa07a934`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:07:35.683228 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-6d78513b-5ad1-480e-b2aa-8c34735737ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:07:35.690218 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ac8784-838e-462a-8708-ec1b45870e32 admin admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:07:35.755874 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8975e491-2e04-4edb-8eb1-d194d3717f39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:07:35.756409 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8975e491-2e04-4edb-8eb1-d194d3717f39 admin admin] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:07:35.756629 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 806/1577] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:07:35 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:07:35.761953 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8975e491-2e04-4edb-8eb1-d194d3717f39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:07:35.761953 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8975e491-2e04-4edb-8eb1-d194d3717f39 admin admin] PATH_INFO: `/v3/projects` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:07:35.762192 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d78513b-5ad1-480e-b2aa-8c34735737ff None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:07:35.762287 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8975e491-2e04-4edb-8eb1-d194d3717f39 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:07:35.762287 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8975e491-2e04-4edb-8eb1-d194d3717f39 admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:07:35.770106 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d78513b-5ad1-480e-b2aa-8c34735737ff None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:07:35.776235 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428375c9-b27b-460a-8089-adb72dd328d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:07:35.776943 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428375c9-b27b-460a-8089-adb72dd328d1 admin admin] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:07:35.777288 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428375c9-b27b-460a-8089-adb72dd328d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:07:35.777623 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428375c9-b27b-460a-8089-adb72dd328d1 admin admin] PATH_INFO: `/v3/users` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:07:35.778293 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428375c9-b27b-460a-8089-adb72dd328d1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:07:35.780341 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428375c9-b27b-460a-8089-adb72dd328d1 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:07:35.843116 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 772/1578] 10.222.0.46 () {60 vars in 1153 bytes} [Mon Oct 10 12:07:35 2022] POST /identity/v3/projects => generated 377 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 12:07:35.843116 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8975e491-2e04-4edb-8eb1-d194d3717f39 admin admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:07:35.868607 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 807/1579] 10.222.0.46 () {60 vars in 1144 bytes} [Mon Oct 10 12:07:35 2022] POST /identity/v3/users => generated 288 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 12:07:35.872701 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428375c9-b27b-460a-8089-adb72dd328d1 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:07:35.880727 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428375c9-b27b-460a-8089-adb72dd328d1 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:07:35.914804 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8975e491-2e04-4edb-8eb1-d194d3717f39 admin admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:07:35.920626 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f221577b-f686-4fc3-841c-b0a427156bea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:07:35.921487 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f221577b-f686-4fc3-841c-b0a427156bea admin admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:07:35.921701 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f221577b-f686-4fc3-841c-b0a427156bea admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:07:35.921978 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f221577b-f686-4fc3-841c-b0a427156bea admin admin] PATH_INFO: `/v3/roles` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:07:35.922717 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f221577b-f686-4fc3-841c-b0a427156bea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:07:35.924616 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f221577b-f686-4fc3-841c-b0a427156bea admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:07:35.936858 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 773/1580] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 10 12:07:35 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 12:07:35.937714 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f221577b-f686-4fc3-841c-b0a427156bea admin admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:07:35.949081 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f221577b-f686-4fc3-841c-b0a427156bea admin admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:07:35.954287 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c4ece1-f926-496a-a8c2-3a619775d2b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:07:35.954973 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c4ece1-f926-496a-a8c2-3a619775d2b6 admin admin] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:07:35.955255 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c4ece1-f926-496a-a8c2-3a619775d2b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:07:35.955532 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c4ece1-f926-496a-a8c2-3a619775d2b6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:07:35.955763 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a083f23-64e7-48f4-ad8e-d1062515e841 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:07:35.956185 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c4ece1-f926-496a-a8c2-3a619775d2b6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:07:35.956679 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a083f23-64e7-48f4-ad8e-d1062515e841 admin admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:07:35.956788 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a083f23-64e7-48f4-ad8e-d1062515e841 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:07:35.957008 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a083f23-64e7-48f4-ad8e-d1062515e841 admin admin] PATH_INFO: `/v3/roles` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:07:35.957671 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a083f23-64e7-48f4-ad8e-d1062515e841 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:07:35.958125 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c4ece1-f926-496a-a8c2-3a619775d2b6 admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:07:35.959606 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a083f23-64e7-48f4-ad8e-d1062515e841 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:07:35.961163 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:07:35.973366 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 808/1581] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 10 12:07:35 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 12:07:35.974073 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a083f23-64e7-48f4-ad8e-d1062515e841 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:07:35.987255 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a083f23-64e7-48f4-ad8e-d1062515e841 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:07:35.993094 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba097f98-d22a-430f-90c8-38f01024a2a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:07:35.993646 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba097f98-d22a-430f-90c8-38f01024a2a8 admin admin] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:07:35.993933 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba097f98-d22a-430f-90c8-38f01024a2a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:07:35.994212 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba097f98-d22a-430f-90c8-38f01024a2a8 admin admin] PATH_INFO: `/v3/users` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:07:35.994909 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba097f98-d22a-430f-90c8-38f01024a2a8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:07:35.996926 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba097f98-d22a-430f-90c8-38f01024a2a8 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:07:36.094247 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 774/1582] 10.222.0.46 () {60 vars in 1153 bytes} [Mon Oct 10 12:07:35 2022] POST /identity/v3/projects => generated 391 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 12:07:36.095303 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c4ece1-f926-496a-a8c2-3a619775d2b6 admin admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:07:36.129312 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 809/1583] 10.222.0.46 () {60 vars in 1144 bytes} [Mon Oct 10 12:07:35 2022] POST /identity/v3/users => generated 295 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 12:07:36.129312 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b871bc9e-75fb-49d9-b4b0-26c3a885db06 None None] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:07:36.129312 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b871bc9e-75fb-49d9-b4b0-26c3a885db06 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:07:36.129312 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b871bc9e-75fb-49d9-b4b0-26c3a885db06 None None] PATH_INFO: `/` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:07:36.130149 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 810/1584] 10.222.0.46 () {54 vars in 961 bytes} [Mon Oct 10 12:07:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 12:07:36.130515 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b871bc9e-75fb-49d9-b4b0-26c3a885db06 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:07:36.138196 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:07:36.169777 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c4ece1-f926-496a-a8c2-3a619775d2b6 admin admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:07:36.174690 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5bcc35-3e10-4514-8ace-ca6fccc03eba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:07:36.175328 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5bcc35-3e10-4514-8ace-ca6fccc03eba admin admin] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:07:36.175614 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5bcc35-3e10-4514-8ace-ca6fccc03eba admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:07:36.176186 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5bcc35-3e10-4514-8ace-ca6fccc03eba admin admin] PATH_INFO: `/v3/users` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:07:36.176955 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5bcc35-3e10-4514-8ace-ca6fccc03eba admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:07:36.178944 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5bcc35-3e10-4514-8ace-ca6fccc03eba admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:07:36.201076 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b871bc9e-75fb-49d9-b4b0-26c3a885db06 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:07:36.205518 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5960fa68-02bd-44b7-920c-7adc1859cdad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:07:36.206120 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5960fa68-02bd-44b7-920c-7adc1859cdad admin admin] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:07:36.206396 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5960fa68-02bd-44b7-920c-7adc1859cdad admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:07:36.206688 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5960fa68-02bd-44b7-920c-7adc1859cdad admin admin] PATH_INFO: `/v3/users` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:07:36.207329 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5960fa68-02bd-44b7-920c-7adc1859cdad admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:07:36.209252 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5960fa68-02bd-44b7-920c-7adc1859cdad admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:07:36.366622 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 775/1585] 10.222.0.46 () {60 vars in 1144 bytes} [Mon Oct 10 12:07:36 2022] POST /identity/v3/users => generated 285 bytes in 272 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 12:07:36.367952 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5bcc35-3e10-4514-8ace-ca6fccc03eba admin admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:07:36.377230 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5bcc35-3e10-4514-8ace-ca6fccc03eba admin admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:07:36.381776 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f22f009-0c8d-4042-8c31-2240a9940597 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:07:36.382350 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f22f009-0c8d-4042-8c31-2240a9940597 admin admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:07:36.382628 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f22f009-0c8d-4042-8c31-2240a9940597 admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:07:36.383092 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f22f009-0c8d-4042-8c31-2240a9940597 admin admin] PATH_INFO: `/v3/roles` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:07:36.383720 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f22f009-0c8d-4042-8c31-2240a9940597 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:07:36.385358 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f22f009-0c8d-4042-8c31-2240a9940597 admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:07:36.396424 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 776/1586] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 10 12:07:36 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 12:07:36.403481 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f22f009-0c8d-4042-8c31-2240a9940597 admin admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:07:36.403624 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 811/1587] 10.222.0.46 () {60 vars in 1144 bytes} [Mon Oct 10 12:07:36 2022] POST /identity/v3/users => generated 289 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 12:07:36.403624 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5960fa68-02bd-44b7-920c-7adc1859cdad admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:07:36.408681 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f22f009-0c8d-4042-8c31-2240a9940597 admin admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:07:36.410368 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5960fa68-02bd-44b7-920c-7adc1859cdad admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:07:36.413763 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552a01ae-c59f-428f-88d4-847b4eb48d95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:07:36.415051 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552a01ae-c59f-428f-88d4-847b4eb48d95 admin admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:07:36.415331 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552a01ae-c59f-428f-88d4-847b4eb48d95 admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:07:36.415600 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552a01ae-c59f-428f-88d4-847b4eb48d95 admin admin] PATH_INFO: `/v3/roles` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:07:36.416085 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a258fcc0-2376-488c-8092-1526576fd063 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:07:36.416226 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552a01ae-c59f-428f-88d4-847b4eb48d95 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:07:36.416741 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a258fcc0-2376-488c-8092-1526576fd063 admin admin] REQUEST_METHOD: `PUT` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:07:36.417040 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a258fcc0-2376-488c-8092-1526576fd063 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:07:36.417375 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a258fcc0-2376-488c-8092-1526576fd063 admin admin] PATH_INFO: `/v3/projects/aab51c9b019e400c8e28d9a789eda6da/users/fef422264d0c4a43a0aa1b9eff2251bd/roles/b2cb2a6e818e47338ce071e6fedfc673` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:07:36.417808 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552a01ae-c59f-428f-88d4-847b4eb48d95 admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:07:36.431746 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 777/1588] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 10 12:07:36 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 12:07:36.431746 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552a01ae-c59f-428f-88d4-847b4eb48d95 admin admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:07:36.438615 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a258fcc0-2376-488c-8092-1526576fd063 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aab51c9b019e400c8e28d9a789eda6da, user_id=fef422264d0c4a43a0aa1b9eff2251bd, role_id=b2cb2a6e818e47338ce071e6fedfc673)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:07:36.438615 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: WARNING py.warnings [None req-a258fcc0-2376-488c-8092-1526576fd063 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 10 12:07:36.438615 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: warnings.warn(msg) Oct 10 12:07:36.438615 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: Oct 10 12:07:36.438615 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a258fcc0-2376-488c-8092-1526576fd063 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:07:36.439206 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552a01ae-c59f-428f-88d4-847b4eb48d95 admin admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:07:36.444006 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0937cc-5fe1-455c-9f2b-fa5647f76e07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:07:36.444604 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0937cc-5fe1-455c-9f2b-fa5647f76e07 admin admin] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:07:36.444917 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0937cc-5fe1-455c-9f2b-fa5647f76e07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:07:36.445325 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0937cc-5fe1-455c-9f2b-fa5647f76e07 admin admin] PATH_INFO: `/v3/users` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:07:36.445927 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0937cc-5fe1-455c-9f2b-fa5647f76e07 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:07:36.448023 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0937cc-5fe1-455c-9f2b-fa5647f76e07 admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:07:36.532550 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 812/1589] 10.222.0.46 () {60 vars in 1483 bytes} [Mon Oct 10 12:07:36 2022] PUT /identity/v3/projects/aab51c9b019e400c8e28d9a789eda6da/users/fef422264d0c4a43a0aa1b9eff2251bd/roles/b2cb2a6e818e47338ce071e6fedfc673 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:07:36.533458 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a258fcc0-2376-488c-8092-1526576fd063 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:07:36.569962 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 778/1590] 10.222.0.46 () {60 vars in 1144 bytes} [Mon Oct 10 12:07:36 2022] POST /identity/v3/users => generated 296 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 12:07:36.569962 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c247d146-e16a-4ba9-9d30-c6fa16a08357 None None] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:07:36.569962 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c247d146-e16a-4ba9-9d30-c6fa16a08357 None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:07:36.569962 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c247d146-e16a-4ba9-9d30-c6fa16a08357 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:07:36.597261 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a258fcc0-2376-488c-8092-1526576fd063 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:07:36.603076 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65882010-a104-4de4-a2f8-f6bfb39907fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:07:36.603699 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65882010-a104-4de4-a2f8-f6bfb39907fe admin admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:07:36.604264 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65882010-a104-4de4-a2f8-f6bfb39907fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:07:36.604623 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65882010-a104-4de4-a2f8-f6bfb39907fe admin admin] PATH_INFO: `/v3/roles` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:07:36.605269 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65882010-a104-4de4-a2f8-f6bfb39907fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:07:36.606904 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65882010-a104-4de4-a2f8-f6bfb39907fe admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:07:36.607430 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.auth.core [None req-c247d146-e16a-4ba9-9d30-c6fa16a08357 None None] MFA Rules not processed for user `5f130bbb69e546ce99b4e14de81d7e54`. Rule list: `[]` (Enabled: `True`). {{(pid=81676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:07:36.608656 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-c247d146-e16a-4ba9-9d30-c6fa16a08357 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:07:36.617335 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 813/1591] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 10 12:07:36 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 12:07:36.618064 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65882010-a104-4de4-a2f8-f6bfb39907fe admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:07:36.627871 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65882010-a104-4de4-a2f8-f6bfb39907fe admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:07:36.632508 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70b75f7-a467-4d9d-bf0b-4015ed6a9da7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:07:36.633192 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70b75f7-a467-4d9d-bf0b-4015ed6a9da7 admin admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:07:36.633476 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70b75f7-a467-4d9d-bf0b-4015ed6a9da7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:07:36.633760 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70b75f7-a467-4d9d-bf0b-4015ed6a9da7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:07:36.634399 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70b75f7-a467-4d9d-bf0b-4015ed6a9da7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:07:36.636021 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70b75f7-a467-4d9d-bf0b-4015ed6a9da7 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:07:36.646858 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 814/1592] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 10 12:07:36 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 12:07:36.647679 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70b75f7-a467-4d9d-bf0b-4015ed6a9da7 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:07:36.660647 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70b75f7-a467-4d9d-bf0b-4015ed6a9da7 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:07:36.666081 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365870dc-00b3-4991-b875-e1a5ce3bb53e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:07:36.666706 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365870dc-00b3-4991-b875-e1a5ce3bb53e admin admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:07:36.666995 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365870dc-00b3-4991-b875-e1a5ce3bb53e admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:07:36.667320 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365870dc-00b3-4991-b875-e1a5ce3bb53e admin admin] PATH_INFO: `/v3/roles` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:07:36.667914 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-365870dc-00b3-4991-b875-e1a5ce3bb53e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:07:36.669608 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-365870dc-00b3-4991-b875-e1a5ce3bb53e admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:07:36.678900 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 779/1593] 10.222.0.46 () {58 vars in 1058 bytes} [Mon Oct 10 12:07:36 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:07:36.679772 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c247d146-e16a-4ba9-9d30-c6fa16a08357 None None] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:07:36.683497 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 815/1594] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 10 12:07:36 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 12:07:36.683497 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365870dc-00b3-4991-b875-e1a5ce3bb53e admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:07:36.691942 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365870dc-00b3-4991-b875-e1a5ce3bb53e admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:07:36.694126 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c247d146-e16a-4ba9-9d30-c6fa16a08357 None None] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:07:36.696832 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0cb9cd-6d28-4790-8520-34e048698696 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:07:36.697504 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0cb9cd-6d28-4790-8520-34e048698696 admin admin] REQUEST_METHOD: `PUT` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:07:36.697789 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0cb9cd-6d28-4790-8520-34e048698696 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:07:36.698094 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0cb9cd-6d28-4790-8520-34e048698696 admin admin] PATH_INFO: `/v3/projects/98437356d32f4230bce856ce6ddae66a/users/b334b2401e14419284cce2ad61442d4a/roles/b2cb2a6e818e47338ce071e6fedfc673` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:07:36.698795 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1719db5f-2b9e-4a46-bfcf-aa25a42db89e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:07:36.699414 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1719db5f-2b9e-4a46-bfcf-aa25a42db89e admin admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:07:36.699688 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1719db5f-2b9e-4a46-bfcf-aa25a42db89e admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:07:36.699961 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1719db5f-2b9e-4a46-bfcf-aa25a42db89e admin admin] PATH_INFO: `/v3/roles` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:07:36.700613 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1719db5f-2b9e-4a46-bfcf-aa25a42db89e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:07:36.702197 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1719db5f-2b9e-4a46-bfcf-aa25a42db89e admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:07:36.715818 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 780/1595] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 10 12:07:36 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 12:07:36.716021 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0cb9cd-6d28-4790-8520-34e048698696 admin admin] RBAC: Authorizing `identity:create_grant(project_id=98437356d32f4230bce856ce6ddae66a, user_id=b334b2401e14419284cce2ad61442d4a, role_id=b2cb2a6e818e47338ce071e6fedfc673)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:07:36.716121 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c7274f-047e-4465-8493-f61cf8f72392 None None] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:07:36.716121 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c7274f-047e-4465-8493-f61cf8f72392 None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:07:36.716121 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c7274f-047e-4465-8493-f61cf8f72392 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:07:36.716571 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0cb9cd-6d28-4790-8520-34e048698696 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:07:36.719042 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:07:36.734436 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.auth.core [None req-14c7274f-047e-4465-8493-f61cf8f72392 None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:07:36.735754 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-14c7274f-047e-4465-8493-f61cf8f72392 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:07:36.743441 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 781/1596] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:07:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:07:36.744212 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c7274f-047e-4465-8493-f61cf8f72392 None None] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:07:36.754045 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c7274f-047e-4465-8493-f61cf8f72392 None None] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:07:36.759107 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779fc4be-91ef-4e03-bfe7-859d83c2179e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:07:36.759728 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779fc4be-91ef-4e03-bfe7-859d83c2179e admin admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:07:36.759989 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779fc4be-91ef-4e03-bfe7-859d83c2179e admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:07:36.760268 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779fc4be-91ef-4e03-bfe7-859d83c2179e admin admin] PATH_INFO: `/v3/roles` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:07:36.760944 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779fc4be-91ef-4e03-bfe7-859d83c2179e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:07:36.762536 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779fc4be-91ef-4e03-bfe7-859d83c2179e admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:07:36.776416 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 782/1597] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 10 12:07:36 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 12:07:36.776416 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779fc4be-91ef-4e03-bfe7-859d83c2179e admin admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:07:36.783856 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779fc4be-91ef-4e03-bfe7-859d83c2179e admin admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:07:36.788671 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26be20fd-8611-4114-8207-c213c91061ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:07:36.789981 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26be20fd-8611-4114-8207-c213c91061ae admin admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:07:36.790270 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26be20fd-8611-4114-8207-c213c91061ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:07:36.790550 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26be20fd-8611-4114-8207-c213c91061ae admin admin] PATH_INFO: `/v3/roles` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:07:36.791225 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26be20fd-8611-4114-8207-c213c91061ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:07:36.792590 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 816/1598] 10.222.0.46 () {60 vars in 1483 bytes} [Mon Oct 10 12:07:36 2022] PUT /identity/v3/projects/98437356d32f4230bce856ce6ddae66a/users/b334b2401e14419284cce2ad61442d4a/roles/b2cb2a6e818e47338ce071e6fedfc673 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:07:36.792810 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26be20fd-8611-4114-8207-c213c91061ae admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:07:36.793428 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0cb9cd-6d28-4790-8520-34e048698696 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:07:36.804217 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 783/1599] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 10 12:07:36 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 12:07:36.805060 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26be20fd-8611-4114-8207-c213c91061ae admin admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:07:36.814738 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26be20fd-8611-4114-8207-c213c91061ae admin admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:07:36.833355 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:07:36.857925 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0cb9cd-6d28-4790-8520-34e048698696 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:07:36.863667 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f3e2f0-ee38-4eb1-90e3-935dc647e7e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:07:36.864452 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f3e2f0-ee38-4eb1-90e3-935dc647e7e6 admin admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:07:36.864736 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f3e2f0-ee38-4eb1-90e3-935dc647e7e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:07:36.865072 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f3e2f0-ee38-4eb1-90e3-935dc647e7e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:07:36.865721 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f3e2f0-ee38-4eb1-90e3-935dc647e7e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:07:36.869922 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f3e2f0-ee38-4eb1-90e3-935dc647e7e6 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:07:36.872790 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c853b46-8dff-4dc5-8b6c-1625b7efd101 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:07:36.873452 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c853b46-8dff-4dc5-8b6c-1625b7efd101 admin admin] REQUEST_METHOD: `PUT` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:07:36.873735 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c853b46-8dff-4dc5-8b6c-1625b7efd101 admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:07:36.874010 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c853b46-8dff-4dc5-8b6c-1625b7efd101 admin admin] PATH_INFO: `/v3/projects/0f034495ee414357a60eb94a0328a037/users/fa9503bf665b42b3a5633eb6e0075f7f/roles/b2cb2a6e818e47338ce071e6fedfc673` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:07:36.877886 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 817/1600] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 10 12:07:36 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 12:07:36.878732 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f3e2f0-ee38-4eb1-90e3-935dc647e7e6 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:07:36.889526 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c853b46-8dff-4dc5-8b6c-1625b7efd101 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0f034495ee414357a60eb94a0328a037, user_id=fa9503bf665b42b3a5633eb6e0075f7f, role_id=b2cb2a6e818e47338ce071e6fedfc673)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:07:36.892230 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c853b46-8dff-4dc5-8b6c-1625b7efd101 admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:07:36.951943 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f3e2f0-ee38-4eb1-90e3-935dc647e7e6 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:07:36.957481 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb3aa0d-5412-420f-9ab0-fd1965b69245 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f130bbb69e546ce99b4e14de81d7e54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad3772f7b6a47c28b7ba37a9316a3c1', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:07:36.958183 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb3aa0d-5412-420f-9ab0-fd1965b69245 service nova] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:07:36.958374 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb3aa0d-5412-420f-9ab0-fd1965b69245 service nova] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:07:36.958650 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb3aa0d-5412-420f-9ab0-fd1965b69245 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:07:36.980849 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 784/1601] 10.222.0.46 () {60 vars in 1483 bytes} [Mon Oct 10 12:07:36 2022] PUT /identity/v3/projects/0f034495ee414357a60eb94a0328a037/users/fa9503bf665b42b3a5633eb6e0075f7f/roles/b2cb2a6e818e47338ce071e6fedfc673 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:07:36.981689 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c853b46-8dff-4dc5-8b6c-1625b7efd101 admin admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:07:37.017503 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb3aa0d-5412-420f-9ab0-fd1965b69245 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:07:37.019450 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb3aa0d-5412-420f-9ab0-fd1965b69245 service nova] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:07:37.029139 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 818/1602] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 10 12:07:36 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:07:37.030034 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb3aa0d-5412-420f-9ab0-fd1965b69245 service nova] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:07:37.052945 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c853b46-8dff-4dc5-8b6c-1625b7efd101 admin admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:07:37.058207 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049d80d8-6bf7-4d07-94ff-2f280658889d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:07:37.059249 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049d80d8-6bf7-4d07-94ff-2f280658889d admin admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:07:37.059541 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049d80d8-6bf7-4d07-94ff-2f280658889d admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:07:37.059681 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:07:37.059830 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049d80d8-6bf7-4d07-94ff-2f280658889d admin admin] PATH_INFO: `/v3/roles` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:07:37.060460 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049d80d8-6bf7-4d07-94ff-2f280658889d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:07:37.062194 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049d80d8-6bf7-4d07-94ff-2f280658889d admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:07:37.075088 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 785/1603] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 10 12:07:36 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 12:07:37.075851 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049d80d8-6bf7-4d07-94ff-2f280658889d admin admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:07:37.086951 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049d80d8-6bf7-4d07-94ff-2f280658889d admin admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:07:37.091838 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd33004-204b-4d31-8a60-420ed38f1829 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:07:37.092574 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd33004-204b-4d31-8a60-420ed38f1829 admin admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:07:37.092860 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd33004-204b-4d31-8a60-420ed38f1829 admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:07:37.093140 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd33004-204b-4d31-8a60-420ed38f1829 admin admin] PATH_INFO: `/v3/roles` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:07:37.093786 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd33004-204b-4d31-8a60-420ed38f1829 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:07:37.095446 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd33004-204b-4d31-8a60-420ed38f1829 admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:07:37.097130 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb3aa0d-5412-420f-9ab0-fd1965b69245 service nova] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:07:37.106268 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 786/1604] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 10 12:07:37 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 12:07:37.107985 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69bf346-1824-428e-b85a-5777cb638ef7 None None] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:07:37.109500 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69bf346-1824-428e-b85a-5777cb638ef7 None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:07:37.109644 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69bf346-1824-428e-b85a-5777cb638ef7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:07:37.130417 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.auth.core [None req-a69bf346-1824-428e-b85a-5777cb638ef7 None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:07:37.131890 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-a69bf346-1824-428e-b85a-5777cb638ef7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:07:37.140441 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 787/1605] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:07:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:07:37.144034 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69bf346-1824-428e-b85a-5777cb638ef7 None None] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:07:37.152184 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69bf346-1824-428e-b85a-5777cb638ef7 None None] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:07:37.157867 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e704b8a0-ba6e-4f4d-b32a-750af2a406ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:07:37.158561 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e704b8a0-ba6e-4f4d-b32a-750af2a406ee admin admin] REQUEST_METHOD: `PUT` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:07:37.158962 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e704b8a0-ba6e-4f4d-b32a-750af2a406ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:07:37.159173 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e704b8a0-ba6e-4f4d-b32a-750af2a406ee admin admin] PATH_INFO: `/v3/projects/af22e9604d79499aa9b7c2ee3cef156d/users/a7a87488f78444c3ad820527e5fbc099/roles/b2cb2a6e818e47338ce071e6fedfc673` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:07:37.159579 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6dfc8c7-dd85-4730-9466-97b5c73108c4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58b1c7c85302421cba823ace4201861f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad3772f7b6a47c28b7ba37a9316a3c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:07:37.160196 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6dfc8c7-dd85-4730-9466-97b5c73108c4 service neutron] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:07:37.160480 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6dfc8c7-dd85-4730-9466-97b5c73108c4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:07:37.160914 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6dfc8c7-dd85-4730-9466-97b5c73108c4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:07:37.172810 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6dfc8c7-dd85-4730-9466-97b5c73108c4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:07:37.174246 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6dfc8c7-dd85-4730-9466-97b5c73108c4 service neutron] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:07:37.177241 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e704b8a0-ba6e-4f4d-b32a-750af2a406ee admin admin] RBAC: Authorizing `identity:create_grant(project_id=af22e9604d79499aa9b7c2ee3cef156d, user_id=a7a87488f78444c3ad820527e5fbc099, role_id=b2cb2a6e818e47338ce071e6fedfc673)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:07:37.180546 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e704b8a0-ba6e-4f4d-b32a-750af2a406ee admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:07:37.185461 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 819/1606] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 10 12:07:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:07:37.186335 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6dfc8c7-dd85-4730-9466-97b5c73108c4 service neutron] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:07:37.196044 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6dfc8c7-dd85-4730-9466-97b5c73108c4 service neutron] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:07:37.200810 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b474308b-3494-49ae-8a33-cebb15ed96cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:07:37.201682 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b474308b-3494-49ae-8a33-cebb15ed96cd admin admin] REQUEST_METHOD: `PUT` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:07:37.201794 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b474308b-3494-49ae-8a33-cebb15ed96cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:07:37.202009 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b474308b-3494-49ae-8a33-cebb15ed96cd admin admin] PATH_INFO: `/v3/projects/65f70a82ea624589925400272b757dea/users/df79e8908130422489d1354663a59b74/roles/b2cb2a6e818e47338ce071e6fedfc673` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:07:37.217042 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b474308b-3494-49ae-8a33-cebb15ed96cd admin admin] RBAC: Authorizing `identity:create_grant(project_id=65f70a82ea624589925400272b757dea, user_id=df79e8908130422489d1354663a59b74, role_id=b2cb2a6e818e47338ce071e6fedfc673)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:07:37.219821 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b474308b-3494-49ae-8a33-cebb15ed96cd admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:07:37.261881 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 788/1607] 10.222.0.46 () {60 vars in 1483 bytes} [Mon Oct 10 12:07:37 2022] PUT /identity/v3/projects/af22e9604d79499aa9b7c2ee3cef156d/users/a7a87488f78444c3ad820527e5fbc099/roles/b2cb2a6e818e47338ce071e6fedfc673 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:07:37.263533 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9e6693-dd94-4b12-b1eb-d5e6cb985c34 None None] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:07:37.264001 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9e6693-dd94-4b12-b1eb-d5e6cb985c34 None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:07:37.264451 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9e6693-dd94-4b12-b1eb-d5e6cb985c34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:07:37.293856 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.auth.core [None req-5c9e6693-dd94-4b12-b1eb-d5e6cb985c34 None None] MFA Rules not processed for user `fa9503bf665b42b3a5633eb6e0075f7f`. Rule list: `[]` (Enabled: `True`). {{(pid=81676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:07:37.295145 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-5c9e6693-dd94-4b12-b1eb-d5e6cb985c34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:07:37.344747 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 820/1608] 10.222.0.46 () {60 vars in 1483 bytes} [Mon Oct 10 12:07:37 2022] PUT /identity/v3/projects/65f70a82ea624589925400272b757dea/users/df79e8908130422489d1354663a59b74/roles/b2cb2a6e818e47338ce071e6fedfc673 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:07:37.346043 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b474308b-3494-49ae-8a33-cebb15ed96cd admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:07:37.381120 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:07:37.418852 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 789/1609] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:07:37 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:07:37.420393 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9e6693-dd94-4b12-b1eb-d5e6cb985c34 None None] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:07:37.449876 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b474308b-3494-49ae-8a33-cebb15ed96cd admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:07:37.456023 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a9f4fc-2015-49e9-a3a3-1911c16e8ebd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:07:37.456946 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a9f4fc-2015-49e9-a3a3-1911c16e8ebd admin admin] REQUEST_METHOD: `PUT` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:07:37.457227 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a9f4fc-2015-49e9-a3a3-1911c16e8ebd admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:07:37.457523 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a9f4fc-2015-49e9-a3a3-1911c16e8ebd admin admin] PATH_INFO: `/v3/projects/f87226283dcb4d6fb203cc5e06cf13b2/users/6bc73e7bcf19443a940b6969e1be8e47/roles/b2cb2a6e818e47338ce071e6fedfc673` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:07:37.475244 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a9f4fc-2015-49e9-a3a3-1911c16e8ebd admin admin] RBAC: Authorizing `identity:create_grant(project_id=f87226283dcb4d6fb203cc5e06cf13b2, user_id=6bc73e7bcf19443a940b6969e1be8e47, role_id=b2cb2a6e818e47338ce071e6fedfc673)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:07:37.477922 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a9f4fc-2015-49e9-a3a3-1911c16e8ebd admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:07:37.504460 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9e6693-dd94-4b12-b1eb-d5e6cb985c34 None None] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:07:37.510283 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84bdb7ca-1611-45c2-8ff0-9f23555d0c20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:07:37.510992 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bdb7ca-1611-45c2-8ff0-9f23555d0c20 admin admin] REQUEST_METHOD: `PUT` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:07:37.511271 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bdb7ca-1611-45c2-8ff0-9f23555d0c20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:07:37.511543 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bdb7ca-1611-45c2-8ff0-9f23555d0c20 admin admin] PATH_INFO: `/v3/projects/782c5fd84235487caac5f9a7e18563cb/users/45c87aa85b7d45fe8acf6f36a3d2a639/roles/b2cb2a6e818e47338ce071e6fedfc673` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:07:37.529438 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84bdb7ca-1611-45c2-8ff0-9f23555d0c20 admin admin] RBAC: Authorizing `identity:create_grant(project_id=782c5fd84235487caac5f9a7e18563cb, user_id=45c87aa85b7d45fe8acf6f36a3d2a639, role_id=b2cb2a6e818e47338ce071e6fedfc673)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:07:37.533428 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84bdb7ca-1611-45c2-8ff0-9f23555d0c20 admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:07:37.587312 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 821/1610] 10.222.0.46 () {60 vars in 1483 bytes} [Mon Oct 10 12:07:37 2022] PUT /identity/v3/projects/f87226283dcb4d6fb203cc5e06cf13b2/users/6bc73e7bcf19443a940b6969e1be8e47/roles/b2cb2a6e818e47338ce071e6fedfc673 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:07:37.587933 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a9f4fc-2015-49e9-a3a3-1911c16e8ebd admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:07:37.626237 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 790/1611] 10.222.0.46 () {60 vars in 1483 bytes} [Mon Oct 10 12:07:37 2022] PUT /identity/v3/projects/782c5fd84235487caac5f9a7e18563cb/users/45c87aa85b7d45fe8acf6f36a3d2a639/roles/b2cb2a6e818e47338ce071e6fedfc673 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:07:37.628258 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11df3ab9-c39e-409b-bc73-721db56639b8 None None] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:07:37.628688 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11df3ab9-c39e-409b-bc73-721db56639b8 None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:07:37.629107 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11df3ab9-c39e-409b-bc73-721db56639b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:07:37.660159 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.auth.core [None req-11df3ab9-c39e-409b-bc73-721db56639b8 None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:07:37.661506 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-11df3ab9-c39e-409b-bc73-721db56639b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:07:37.662086 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:07:37.676637 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a9f4fc-2015-49e9-a3a3-1911c16e8ebd admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:07:37.736079 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 791/1612] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:07:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:07:37.747179 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8cb1de-9180-4f06-ae14-9fc819359774 None None] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:07:37.747179 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8cb1de-9180-4f06-ae14-9fc819359774 None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:07:37.747179 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8cb1de-9180-4f06-ae14-9fc819359774 None None] PATH_INFO: `/` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:07:37.750057 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 792/1613] 10.222.0.46 () {54 vars in 912 bytes} [Mon Oct 10 12:07:37 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 12:07:37.752553 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e94574-f16d-4125-951f-bdf2f191525a None None] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:07:37.753456 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e94574-f16d-4125-951f-bdf2f191525a None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:07:37.753914 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e94574-f16d-4125-951f-bdf2f191525a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:07:37.765177 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9d6bd4-9c98-451a-9196-4a69b474f292 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58b1c7c85302421cba823ace4201861f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad3772f7b6a47c28b7ba37a9316a3c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:07:37.766174 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9d6bd4-9c98-451a-9196-4a69b474f292 service neutron] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:07:37.766651 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9d6bd4-9c98-451a-9196-4a69b474f292 service neutron] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:07:37.766984 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9d6bd4-9c98-451a-9196-4a69b474f292 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:07:37.778717 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9d6bd4-9c98-451a-9196-4a69b474f292 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:07:37.780279 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9d6bd4-9c98-451a-9196-4a69b474f292 service neutron] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:07:37.795903 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 822/1614] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 10 12:07:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 208 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:07:37.798155 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20727671-3cbc-47e3-84f9-e9d9e4f6c03c None None] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:07:37.798608 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20727671-3cbc-47e3-84f9-e9d9e4f6c03c None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:07:37.799081 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20727671-3cbc-47e3-84f9-e9d9e4f6c03c None None] PATH_INFO: `/` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:07:37.801727 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 823/1615] 10.222.0.46 () {54 vars in 961 bytes} [Mon Oct 10 12:07:37 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 12:07:37.811984 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fb73e8-fd0d-4598-a1b7-562a5ef487b8 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:07:37.812775 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fb73e8-fd0d-4598-a1b7-562a5ef487b8 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:07:37.813486 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fb73e8-fd0d-4598-a1b7-562a5ef487b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:07:37.813670 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.auth.core [None req-52e94574-f16d-4125-951f-bdf2f191525a None None] MFA Rules not processed for user `df79e8908130422489d1354663a59b74`. Rule list: `[]` (Enabled: `True`). {{(pid=81676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:07:37.816073 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-52e94574-f16d-4125-951f-bdf2f191525a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:07:37.846607 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-71fb73e8-fd0d-4598-a1b7-562a5ef487b8 None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:07:37.848926 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-71fb73e8-fd0d-4598-a1b7-562a5ef487b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:07:37.872855 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 824/1616] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:07:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:07:37.875088 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729fcd97-cc7d-4d94-995b-3e52be86491e None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:07:37.875500 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729fcd97-cc7d-4d94-995b-3e52be86491e None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:07:37.876367 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729fcd97-cc7d-4d94-995b-3e52be86491e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:07:37.909831 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-729fcd97-cc7d-4d94-995b-3e52be86491e None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:07:37.911296 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-729fcd97-cc7d-4d94-995b-3e52be86491e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:07:37.920369 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 825/1617] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:07:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:07:37.925051 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4845f0b2-8458-4794-a6bc-7f513514bd8b None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:07:37.925348 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4845f0b2-8458-4794-a6bc-7f513514bd8b None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:07:37.925628 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4845f0b2-8458-4794-a6bc-7f513514bd8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:07:37.929055 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:07:37.934293 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 793/1618] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:07:37 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:07:37.935410 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e94574-f16d-4125-951f-bdf2f191525a None None] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:07:37.949716 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e94574-f16d-4125-951f-bdf2f191525a None None] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:07:37.953140 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-4845f0b2-8458-4794-a6bc-7f513514bd8b None None] MFA Rules not processed for user `5f130bbb69e546ce99b4e14de81d7e54`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:07:37.956512 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b5a577-cd83-4f7f-b12c-943d1b503b6c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58b1c7c85302421cba823ace4201861f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad3772f7b6a47c28b7ba37a9316a3c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:07:37.956758 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-4845f0b2-8458-4794-a6bc-7f513514bd8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:07:37.956936 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b5a577-cd83-4f7f-b12c-943d1b503b6c service neutron] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:07:37.956936 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b5a577-cd83-4f7f-b12c-943d1b503b6c service neutron] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:07:37.956936 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b5a577-cd83-4f7f-b12c-943d1b503b6c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:07:37.967188 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b5a577-cd83-4f7f-b12c-943d1b503b6c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:07:37.969124 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b5a577-cd83-4f7f-b12c-943d1b503b6c service neutron] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:07:37.981857 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 794/1619] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 10 12:07:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:07:37.987262 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc30ffd-f7fe-405d-bb3d-82ef91a31df9 None None] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:07:37.987262 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc30ffd-f7fe-405d-bb3d-82ef91a31df9 None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:07:37.987262 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc30ffd-f7fe-405d-bb3d-82ef91a31df9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:07:38.009087 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.auth.core [None req-2fc30ffd-f7fe-405d-bb3d-82ef91a31df9 None None] MFA Rules not processed for user `5f130bbb69e546ce99b4e14de81d7e54`. Rule list: `[]` (Enabled: `True`). {{(pid=81676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:07:38.015382 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-2fc30ffd-f7fe-405d-bb3d-82ef91a31df9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:07:38.075329 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 826/1620] 10.222.0.46 () {58 vars in 1009 bytes} [Mon Oct 10 12:07:37 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:07:38.076444 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4845f0b2-8458-4794-a6bc-7f513514bd8b None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:07:38.089143 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4845f0b2-8458-4794-a6bc-7f513514bd8b None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:07:38.095154 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e18e75-c306-40aa-bf1f-8d5a2b722495 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58b1c7c85302421cba823ace4201861f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad3772f7b6a47c28b7ba37a9316a3c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:07:38.095821 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e18e75-c306-40aa-bf1f-8d5a2b722495 service neutron] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:07:38.096339 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e18e75-c306-40aa-bf1f-8d5a2b722495 service neutron] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:07:38.096715 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e18e75-c306-40aa-bf1f-8d5a2b722495 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:07:38.110199 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e18e75-c306-40aa-bf1f-8d5a2b722495 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:07:38.112197 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e18e75-c306-40aa-bf1f-8d5a2b722495 service neutron] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:07:38.124638 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 827/1621] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 10 12:07:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:07:38.125472 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e18e75-c306-40aa-bf1f-8d5a2b722495 service neutron] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:07:38.135350 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e18e75-c306-40aa-bf1f-8d5a2b722495 service neutron] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:07:38.136472 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 795/1622] 10.222.0.46 () {58 vars in 1058 bytes} [Mon Oct 10 12:07:37 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:07:38.140102 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629b49cb-c8a3-41c3-b554-aeed68772088 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58b1c7c85302421cba823ace4201861f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad3772f7b6a47c28b7ba37a9316a3c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:07:38.140737 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629b49cb-c8a3-41c3-b554-aeed68772088 service neutron] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:07:38.141333 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629b49cb-c8a3-41c3-b554-aeed68772088 service neutron] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:07:38.141610 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629b49cb-c8a3-41c3-b554-aeed68772088 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:07:38.144061 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc30ffd-f7fe-405d-bb3d-82ef91a31df9 None None] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:07:38.153902 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629b49cb-c8a3-41c3-b554-aeed68772088 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:07:38.155696 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629b49cb-c8a3-41c3-b554-aeed68772088 service neutron] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:07:38.167970 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc30ffd-f7fe-405d-bb3d-82ef91a31df9 None None] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:07:38.168174 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 828/1623] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 10 12:07:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:07:38.168649 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7493792-8294-4f0c-a19d-9fab3d3913e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:07:38.169084 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7493792-8294-4f0c-a19d-9fab3d3913e6 admin admin] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:07:38.169334 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629b49cb-c8a3-41c3-b554-aeed68772088 service neutron] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:07:38.169714 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7493792-8294-4f0c-a19d-9fab3d3913e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:07:38.169969 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7493792-8294-4f0c-a19d-9fab3d3913e6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:07:38.171603 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7493792-8294-4f0c-a19d-9fab3d3913e6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:07:38.174464 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7493792-8294-4f0c-a19d-9fab3d3913e6 admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:07:38.277253 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629b49cb-c8a3-41c3-b554-aeed68772088 service neutron] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:07:38.282462 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d475da4-2df5-4118-87ad-c4e7096a8a3b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3ae093470484b93b23dcfae1657df9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad3772f7b6a47c28b7ba37a9316a3c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:07:38.283351 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d475da4-2df5-4118-87ad-c4e7096a8a3b service cinder] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:07:38.283550 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d475da4-2df5-4118-87ad-c4e7096a8a3b service cinder] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:07:38.283937 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d475da4-2df5-4118-87ad-c4e7096a8a3b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:07:38.296860 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 796/1624] 10.222.0.46 () {60 vars in 1153 bytes} [Mon Oct 10 12:07:38 2022] POST /identity/v3/projects => generated 369 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 12:07:38.296860 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7493792-8294-4f0c-a19d-9fab3d3913e6 admin admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:07:38.297655 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d475da4-2df5-4118-87ad-c4e7096a8a3b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:07:38.298799 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d475da4-2df5-4118-87ad-c4e7096a8a3b service cinder] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:07:38.303817 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:07:38.311852 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 829/1625] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 10 12:07:38 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:07:38.312613 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d475da4-2df5-4118-87ad-c4e7096a8a3b service cinder] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:07:38.323498 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d475da4-2df5-4118-87ad-c4e7096a8a3b service cinder] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:07:38.401758 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b58749f-a89c-4db5-9a71-0e754e6f6b3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:07:38.402384 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b58749f-a89c-4db5-9a71-0e754e6f6b3c admin admin] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:07:38.402718 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b58749f-a89c-4db5-9a71-0e754e6f6b3c admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:07:38.403426 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b58749f-a89c-4db5-9a71-0e754e6f6b3c admin admin] PATH_INFO: `/v3/users` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:07:38.410036 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b58749f-a89c-4db5-9a71-0e754e6f6b3c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:07:38.410159 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7493792-8294-4f0c-a19d-9fab3d3913e6 admin admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:07:38.410281 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b58749f-a89c-4db5-9a71-0e754e6f6b3c admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:07:38.425537 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7643715c-9528-4ecf-adf4-53e3b2ff8aa5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f130bbb69e546ce99b4e14de81d7e54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad3772f7b6a47c28b7ba37a9316a3c1', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:07:38.426656 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7643715c-9528-4ecf-adf4-53e3b2ff8aa5 service nova] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:07:38.427064 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7643715c-9528-4ecf-adf4-53e3b2ff8aa5 service nova] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:07:38.428342 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7643715c-9528-4ecf-adf4-53e3b2ff8aa5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:07:38.498723 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 830/1626] 10.222.0.46 () {60 vars in 1144 bytes} [Mon Oct 10 12:07:38 2022] POST /identity/v3/users => generated 291 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 12:07:38.501856 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b58749f-a89c-4db5-9a71-0e754e6f6b3c admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:07:38.520412 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b58749f-a89c-4db5-9a71-0e754e6f6b3c admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:07:38.533185 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe91a24-bae6-42e8-a4fb-51c51930f032 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:07:38.533185 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe91a24-bae6-42e8-a4fb-51c51930f032 admin admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:07:38.533185 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe91a24-bae6-42e8-a4fb-51c51930f032 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:07:38.535668 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:07:38.542976 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe91a24-bae6-42e8-a4fb-51c51930f032 admin admin] PATH_INFO: `/v3/roles` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:07:38.543397 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7643715c-9528-4ecf-adf4-53e3b2ff8aa5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:07:38.544547 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe91a24-bae6-42e8-a4fb-51c51930f032 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:07:38.546761 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7643715c-9528-4ecf-adf4-53e3b2ff8aa5 service nova] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:07:38.550444 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe91a24-bae6-42e8-a4fb-51c51930f032 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:07:38.562724 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 797/1627] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 10 12:07:38 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 268 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:07:38.575632 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 831/1628] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 10 12:07:38 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 12:07:38.579784 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7643715c-9528-4ecf-adf4-53e3b2ff8aa5 service nova] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:07:38.592129 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7643715c-9528-4ecf-adf4-53e3b2ff8aa5 service nova] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:07:38.597601 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e165127a-1957-463d-ac61-6aa68ffea4bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:07:38.598652 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e165127a-1957-463d-ac61-6aa68ffea4bc admin admin] REQUEST_METHOD: `PUT` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:07:38.599193 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e165127a-1957-463d-ac61-6aa68ffea4bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:07:38.599793 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e165127a-1957-463d-ac61-6aa68ffea4bc admin admin] PATH_INFO: `/v3/projects/abfd589aa65542ca96d05012a56664a8/users/a8ddba03273842d3b41ef01cd904e0b1/roles/0d822a0674b6495fac623877a3b38025` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:07:38.631149 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e165127a-1957-463d-ac61-6aa68ffea4bc admin admin] RBAC: Authorizing `identity:create_grant(project_id=abfd589aa65542ca96d05012a56664a8, user_id=a8ddba03273842d3b41ef01cd904e0b1, role_id=0d822a0674b6495fac623877a3b38025)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:07:38.635962 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e165127a-1957-463d-ac61-6aa68ffea4bc admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:07:38.777639 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 798/1629] 10.222.0.46 () {60 vars in 1483 bytes} [Mon Oct 10 12:07:38 2022] PUT /identity/v3/projects/abfd589aa65542ca96d05012a56664a8/users/a8ddba03273842d3b41ef01cd904e0b1/roles/0d822a0674b6495fac623877a3b38025 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:07:38.778960 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe91a24-bae6-42e8-a4fb-51c51930f032 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:07:38.799004 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe91a24-bae6-42e8-a4fb-51c51930f032 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:07:38.919446 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c75c966-54e4-4be1-936d-7e98baf5472a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:07:38.920185 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c75c966-54e4-4be1-936d-7e98baf5472a admin admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:07:38.920479 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c75c966-54e4-4be1-936d-7e98baf5472a admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:07:38.920759 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c75c966-54e4-4be1-936d-7e98baf5472a admin admin] PATH_INFO: `/v3/roles` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:07:38.921665 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c75c966-54e4-4be1-936d-7e98baf5472a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:07:38.923348 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c75c966-54e4-4be1-936d-7e98baf5472a admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:07:38.950079 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 832/1630] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 10 12:07:38 2022] GET /identity/v3/roles => generated 1420 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 12:07:38.950375 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e165127a-1957-463d-ac61-6aa68ffea4bc admin admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:07:38.979298 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e165127a-1957-463d-ac61-6aa68ffea4bc admin admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:07:38.985641 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34aff0eb-2705-4519-a026-eeeeb17eb292 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:07:38.986493 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34aff0eb-2705-4519-a026-eeeeb17eb292 admin admin] REQUEST_METHOD: `PUT` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:07:38.986832 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34aff0eb-2705-4519-a026-eeeeb17eb292 admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:07:38.987117 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34aff0eb-2705-4519-a026-eeeeb17eb292 admin admin] PATH_INFO: `/v3/projects/abfd589aa65542ca96d05012a56664a8/users/a8ddba03273842d3b41ef01cd904e0b1/roles/0d822a0674b6495fac623877a3b38025` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:07:38.989276 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34aff0eb-2705-4519-a026-eeeeb17eb292 admin admin] RBAC: Authorizing `identity:create_grant(project_id=abfd589aa65542ca96d05012a56664a8, user_id=a8ddba03273842d3b41ef01cd904e0b1, role_id=0d822a0674b6495fac623877a3b38025)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:07:38.993676 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34aff0eb-2705-4519-a026-eeeeb17eb292 admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:07:39.009411 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 799/1631] 10.222.0.46 () {60 vars in 1483 bytes} [Mon Oct 10 12:07:38 2022] PUT /identity/v3/projects/abfd589aa65542ca96d05012a56664a8/users/a8ddba03273842d3b41ef01cd904e0b1/roles/0d822a0674b6495fac623877a3b38025 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:07:39.020605 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd75c91-e24d-4cd0-abc2-a76391305f55 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:07:39.021516 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd75c91-e24d-4cd0-abc2-a76391305f55 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:07:39.022229 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd75c91-e24d-4cd0-abc2-a76391305f55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:07:39.297808 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-2fd75c91-e24d-4cd0-abc2-a76391305f55 None None] MFA Rules not processed for user `a8ddba03273842d3b41ef01cd904e0b1`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:07:39.297808 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-2fd75c91-e24d-4cd0-abc2-a76391305f55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:07:39.327977 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4323016e-db60-478a-88e6-ba8ab909c8b1 None None] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:07:39.327977 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4323016e-db60-478a-88e6-ba8ab909c8b1 None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:07:39.327977 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4323016e-db60-478a-88e6-ba8ab909c8b1 None None] PATH_INFO: `/` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:07:39.327977 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 800/1632] 10.222.0.46 () {54 vars in 912 bytes} [Mon Oct 10 12:07:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 12:07:39.327977 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950c86fb-72e9-4c35-ac11-bd6b0e4051de None None] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:07:39.327977 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950c86fb-72e9-4c35-ac11-bd6b0e4051de None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:07:39.335247 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950c86fb-72e9-4c35-ac11-bd6b0e4051de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:07:39.339525 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:07:39.339525 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 833/1633] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:07:39 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:07:39.343277 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.auth.core [None req-950c86fb-72e9-4c35-ac11-bd6b0e4051de None None] MFA Rules not processed for user `5f130bbb69e546ce99b4e14de81d7e54`. Rule list: `[]` (Enabled: `True`). {{(pid=81676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:07:39.343277 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-950c86fb-72e9-4c35-ac11-bd6b0e4051de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:07:39.356430 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd75c91-e24d-4cd0-abc2-a76391305f55 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:07:39.356430 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd75c91-e24d-4cd0-abc2-a76391305f55 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:07:39.356430 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344381cb-19dc-4b4d-8485-2cd16aa1beb9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3ae093470484b93b23dcfae1657df9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad3772f7b6a47c28b7ba37a9316a3c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:07:39.356430 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344381cb-19dc-4b4d-8485-2cd16aa1beb9 service cinder] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:07:39.368103 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344381cb-19dc-4b4d-8485-2cd16aa1beb9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:07:39.368103 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344381cb-19dc-4b4d-8485-2cd16aa1beb9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:07:39.385194 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344381cb-19dc-4b4d-8485-2cd16aa1beb9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:07:39.385194 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344381cb-19dc-4b4d-8485-2cd16aa1beb9 service cinder] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:07:39.412465 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 834/1634] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 10 12:07:39 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 208 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:07:39.450745 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 801/1635] 10.222.0.46 () {58 vars in 1009 bytes} [Mon Oct 10 12:07:39 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 274 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 10 12:07:39.590876 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344381cb-19dc-4b4d-8485-2cd16aa1beb9 service cinder] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:07:39.606360 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344381cb-19dc-4b4d-8485-2cd16aa1beb9 service cinder] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:07:39.614209 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1015ab-6ac8-4e0c-9b12-4570721d660b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3ae093470484b93b23dcfae1657df9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad3772f7b6a47c28b7ba37a9316a3c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:07:39.614816 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1015ab-6ac8-4e0c-9b12-4570721d660b service cinder] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:07:39.615284 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1015ab-6ac8-4e0c-9b12-4570721d660b service cinder] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:07:39.615633 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1015ab-6ac8-4e0c-9b12-4570721d660b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:07:39.640913 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1015ab-6ac8-4e0c-9b12-4570721d660b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:07:39.643578 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1015ab-6ac8-4e0c-9b12-4570721d660b service cinder] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:07:39.670703 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 835/1636] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 10 12:07:39 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:07:39.712229 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950c86fb-72e9-4c35-ac11-bd6b0e4051de None None] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:07:39.732857 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950c86fb-72e9-4c35-ac11-bd6b0e4051de None None] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:07:39.740427 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c59b1c6-37df-4c69-980d-e028c89ccb48 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3ae093470484b93b23dcfae1657df9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad3772f7b6a47c28b7ba37a9316a3c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:07:39.741072 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c59b1c6-37df-4c69-980d-e028c89ccb48 service cinder] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:07:39.741646 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c59b1c6-37df-4c69-980d-e028c89ccb48 service cinder] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:07:39.742088 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c59b1c6-37df-4c69-980d-e028c89ccb48 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:07:39.914444 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c59b1c6-37df-4c69-980d-e028c89ccb48 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:07:39.916046 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c59b1c6-37df-4c69-980d-e028c89ccb48 service cinder] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:07:39.944047 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 802/1637] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 10 12:07:39 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 233 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:07:40.319700 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1015ab-6ac8-4e0c-9b12-4570721d660b service cinder] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:07:40.473026 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1015ab-6ac8-4e0c-9b12-4570721d660b service cinder] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:07:40.484628 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd52094-bc84-4c1b-b141-b2d8bb6ff550 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:07:40.485582 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd52094-bc84-4c1b-b141-b2d8bb6ff550 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:07:40.486708 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd52094-bc84-4c1b-b141-b2d8bb6ff550 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:07:40.487627 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd52094-bc84-4c1b-b141-b2d8bb6ff550 admin admin] PATH_INFO: `/v3/users/fa9503bf665b42b3a5633eb6e0075f7f` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:07:40.490116 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd52094-bc84-4c1b-b141-b2d8bb6ff550 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fa9503bf665b42b3a5633eb6e0075f7f)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:07:40.491197 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: WARNING py.warnings [None req-1cd52094-bc84-4c1b-b141-b2d8bb6ff550 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 10 12:07:40.491197 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: warnings.warn(msg) Oct 10 12:07:40.491197 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: Oct 10 12:07:40.493761 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd52094-bc84-4c1b-b141-b2d8bb6ff550 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:07:40.793779 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-1cd52094-bc84-4c1b-b141-b2d8bb6ff550 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fa9503bf665b42b3a5633eb6e0075f7f'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:07:40.814980 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-1cd52094-bc84-4c1b-b141-b2d8bb6ff550 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fa9503bf665b42b3a5633eb6e0075f7f'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:07:40.816657 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-1cd52094-bc84-4c1b-b141-b2d8bb6ff550 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fa9503bf665b42b3a5633eb6e0075f7f'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:07:40.957593 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-1cd52094-bc84-4c1b-b141-b2d8bb6ff550 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fa9503bf665b42b3a5633eb6e0075f7f'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:07:40.999391 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-1cd52094-bc84-4c1b-b141-b2d8bb6ff550 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fa9503bf665b42b3a5633eb6e0075f7f'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:07:41.022203 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 836/1638] 10.222.0.46 () {58 vars in 1224 bytes} [Mon Oct 10 12:07:40 2022] DELETE /identity/v3/users/fa9503bf665b42b3a5633eb6e0075f7f => generated 0 bytes in 704 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:07:41.027514 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4decaf-9f0f-4555-9a87-3a630b8f898b None None] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:07:41.028093 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4decaf-9f0f-4555-9a87-3a630b8f898b None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:07:41.028093 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4decaf-9f0f-4555-9a87-3a630b8f898b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:07:41.062338 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.auth.core [None req-ea4decaf-9f0f-4555-9a87-3a630b8f898b None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:07:41.063695 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-ea4decaf-9f0f-4555-9a87-3a630b8f898b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:07:41.193856 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:07:41.233248 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 803/1639] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:07:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 10 12:07:41.312397 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd52094-bc84-4c1b-b141-b2d8bb6ff550 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:07:41.314017 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-1cd52094-bc84-4c1b-b141-b2d8bb6ff550 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:07:41.334370 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:07:41.497771 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd52094-bc84-4c1b-b141-b2d8bb6ff550 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:07:41.506294 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151a8928-1469-41e7-b843-77145369cb93 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58b1c7c85302421cba823ace4201861f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad3772f7b6a47c28b7ba37a9316a3c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:07:41.509291 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151a8928-1469-41e7-b843-77145369cb93 service neutron] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:07:41.510465 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151a8928-1469-41e7-b843-77145369cb93 service neutron] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:07:41.511239 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151a8928-1469-41e7-b843-77145369cb93 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:07:41.528393 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151a8928-1469-41e7-b843-77145369cb93 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:07:41.532441 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151a8928-1469-41e7-b843-77145369cb93 service neutron] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:07:41.549957 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 837/1640] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 10 12:07:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 238 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:07:44.241667 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4decaf-9f0f-4555-9a87-3a630b8f898b None None] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:07:44.244005 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-ea4decaf-9f0f-4555-9a87-3a630b8f898b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:07:44.259831 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4decaf-9f0f-4555-9a87-3a630b8f898b None None] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:07:44.265967 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef49b24f-6364-4fc9-b417-0b9b633cbdb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:07:44.266651 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef49b24f-6364-4fc9-b417-0b9b633cbdb1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:07:44.266966 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef49b24f-6364-4fc9-b417-0b9b633cbdb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:07:44.267335 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef49b24f-6364-4fc9-b417-0b9b633cbdb1 admin admin] PATH_INFO: `/v3/projects/0f034495ee414357a60eb94a0328a037` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:07:44.268689 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef49b24f-6364-4fc9-b417-0b9b633cbdb1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0f034495ee414357a60eb94a0328a037)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:07:44.269773 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: WARNING py.warnings [None req-ef49b24f-6364-4fc9-b417-0b9b633cbdb1 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 10 12:07:44.269773 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: warnings.warn(msg) Oct 10 12:07:44.269773 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: Oct 10 12:07:44.272311 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef49b24f-6364-4fc9-b417-0b9b633cbdb1 admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:07:44.362714 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151a8928-1469-41e7-b843-77145369cb93 service neutron] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:07:44.365221 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-151a8928-1469-41e7-b843-77145369cb93 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:07:44.382318 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151a8928-1469-41e7-b843-77145369cb93 service neutron] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:07:44.387676 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ff1096-00b2-4ea6-91d6-4892452003ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:07:44.388707 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ff1096-00b2-4ea6-91d6-4892452003ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:07:44.389142 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ff1096-00b2-4ea6-91d6-4892452003ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:07:44.389587 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ff1096-00b2-4ea6-91d6-4892452003ae admin admin] PATH_INFO: `/v3/users/d30a538ba96147718bbdcb6bfa07a934` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:07:44.391131 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ff1096-00b2-4ea6-91d6-4892452003ae admin admin] RBAC: Authorizing `identity:delete_user(user_id=d30a538ba96147718bbdcb6bfa07a934)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:07:44.393881 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ff1096-00b2-4ea6-91d6-4892452003ae admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:07:44.439853 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-ef49b24f-6364-4fc9-b417-0b9b633cbdb1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0f034495ee414357a60eb94a0328a037', 'request_id': 'req-ef49b24f-6364-4fc9-b417-0b9b633cbdb1'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:07:44.477517 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-ef49b24f-6364-4fc9-b417-0b9b633cbdb1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0f034495ee414357a60eb94a0328a037', 'request_id': 'req-ef49b24f-6364-4fc9-b417-0b9b633cbdb1'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:07:44.530279 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-ef49b24f-6364-4fc9-b417-0b9b633cbdb1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0f034495ee414357a60eb94a0328a037', 'request_id': 'req-ef49b24f-6364-4fc9-b417-0b9b633cbdb1'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:07:44.645874 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-ef49b24f-6364-4fc9-b417-0b9b633cbdb1 admin admin] The token cache is being invalidate because project 0f034495ee414357a60eb94a0328a037 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81676) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 10 12:07:44.646170 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-ef49b24f-6364-4fc9-b417-0b9b633cbdb1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:07:44.648817 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-ef49b24f-6364-4fc9-b417-0b9b633cbdb1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:07:44.652068 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 804/1641] 10.222.0.46 () {58 vars in 1233 bytes} [Mon Oct 10 12:07:44 2022] DELETE /identity/v3/projects/0f034495ee414357a60eb94a0328a037 => generated 0 bytes in 411 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:07:44.656988 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c790fbe-b295-4de0-badf-d80d271e6b73 None None] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:07:44.657417 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c790fbe-b295-4de0-badf-d80d271e6b73 None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:07:44.664657 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c790fbe-b295-4de0-badf-d80d271e6b73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:07:44.739050 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-a1ff1096-00b2-4ea6-91d6-4892452003ae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd30a538ba96147718bbdcb6bfa07a934'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:07:44.764100 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-a1ff1096-00b2-4ea6-91d6-4892452003ae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd30a538ba96147718bbdcb6bfa07a934'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:07:44.765192 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-a1ff1096-00b2-4ea6-91d6-4892452003ae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd30a538ba96147718bbdcb6bfa07a934'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:07:44.774370 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.auth.core [None req-9c790fbe-b295-4de0-badf-d80d271e6b73 None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:07:44.780509 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-9c790fbe-b295-4de0-badf-d80d271e6b73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:07:44.920328 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-a1ff1096-00b2-4ea6-91d6-4892452003ae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd30a538ba96147718bbdcb6bfa07a934'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:07:44.982090 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-a1ff1096-00b2-4ea6-91d6-4892452003ae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd30a538ba96147718bbdcb6bfa07a934'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:07:45.012055 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 838/1642] 10.222.0.46 () {58 vars in 1224 bytes} [Mon Oct 10 12:07:44 2022] DELETE /identity/v3/users/d30a538ba96147718bbdcb6bfa07a934 => generated 0 bytes in 650 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:07:45.020380 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacd0f8c-c426-430d-af14-7ddb1b1f1308 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:07:45.020917 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacd0f8c-c426-430d-af14-7ddb1b1f1308 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:07:45.022115 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacd0f8c-c426-430d-af14-7ddb1b1f1308 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:07:45.035147 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 805/1643] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:07:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 379 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:07:45.044327 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b848cafa-a190-4e59-848b-1c4c793f9563 None None] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:07:45.044713 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b848cafa-a190-4e59-848b-1c4c793f9563 None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:07:45.045190 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b848cafa-a190-4e59-848b-1c4c793f9563 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:07:45.074331 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.auth.core [None req-b848cafa-a190-4e59-848b-1c4c793f9563 None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:07:45.076037 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-b848cafa-a190-4e59-848b-1c4c793f9563 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:07:45.080453 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-cacd0f8c-c426-430d-af14-7ddb1b1f1308 None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:07:45.085013 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-cacd0f8c-c426-430d-af14-7ddb1b1f1308 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:07:45.208992 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 806/1644] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:07:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:07:45.211316 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b848cafa-a190-4e59-848b-1c4c793f9563 None None] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:07:45.235611 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b848cafa-a190-4e59-848b-1c4c793f9563 None None] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:07:45.250778 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 839/1645] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:07:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:07:45.262830 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed63d61-6d41-4eec-9227-391e1fc055ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:07:45.263486 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed63d61-6d41-4eec-9227-391e1fc055ff admin admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:07:45.263776 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed63d61-6d41-4eec-9227-391e1fc055ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:07:45.264034 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed63d61-6d41-4eec-9227-391e1fc055ff admin admin] PATH_INFO: `/v3/domains` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:07:45.264720 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed63d61-6d41-4eec-9227-391e1fc055ff admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 12:07:45.265037 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed63d61-6d41-4eec-9227-391e1fc055ff admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:07:45.268978 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed63d61-6d41-4eec-9227-391e1fc055ff admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:07:45.280299 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:07:45.294416 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cacd0f8c-c426-430d-af14-7ddb1b1f1308 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:07:45.302805 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 807/1646] 10.222.0.46 () {58 vars in 1153 bytes} [Mon Oct 10 12:07:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 12:07:45.312093 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cacd0f8c-c426-430d-af14-7ddb1b1f1308 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:07:45.327856 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2babe0d-0b42-4134-a8d8-1fefcea7e700 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:07:45.328507 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2babe0d-0b42-4134-a8d8-1fefcea7e700 admin admin] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:07:45.328789 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2babe0d-0b42-4134-a8d8-1fefcea7e700 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:07:45.329061 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2babe0d-0b42-4134-a8d8-1fefcea7e700 admin admin] PATH_INFO: `/v3/projects` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:07:45.329709 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2babe0d-0b42-4134-a8d8-1fefcea7e700 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:07:45.336645 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2babe0d-0b42-4134-a8d8-1fefcea7e700 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:07:45.443969 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:07:45.457795 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 840/1647] 10.222.0.46 () {60 vars in 1153 bytes} [Mon Oct 10 12:07:45 2022] POST /identity/v3/projects => generated 363 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 12:07:45.459844 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed63d61-6d41-4eec-9227-391e1fc055ff admin admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:07:45.474138 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed63d61-6d41-4eec-9227-391e1fc055ff admin admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:07:45.505888 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2babe0d-0b42-4134-a8d8-1fefcea7e700 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:07:45.507904 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-d2babe0d-0b42-4134-a8d8-1fefcea7e700 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:07:45.647589 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2babe0d-0b42-4134-a8d8-1fefcea7e700 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:07:45.655953 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4dd3555-17b3-4bd5-92d9-605bd153a536 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58b1c7c85302421cba823ace4201861f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad3772f7b6a47c28b7ba37a9316a3c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:07:45.656593 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4dd3555-17b3-4bd5-92d9-605bd153a536 service neutron] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:07:45.656872 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4dd3555-17b3-4bd5-92d9-605bd153a536 service neutron] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:07:45.657146 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4dd3555-17b3-4bd5-92d9-605bd153a536 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:07:45.659459 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9ba7e1-4521-4b6d-a50d-cd69de85ac80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:07:45.660008 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9ba7e1-4521-4b6d-a50d-cd69de85ac80 admin admin] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:07:45.660282 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9ba7e1-4521-4b6d-a50d-cd69de85ac80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:07:45.660551 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9ba7e1-4521-4b6d-a50d-cd69de85ac80 admin admin] PATH_INFO: `/v3/users` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:07:45.661188 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9ba7e1-4521-4b6d-a50d-cd69de85ac80 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:07:45.663174 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9ba7e1-4521-4b6d-a50d-cd69de85ac80 admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:07:45.680672 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4dd3555-17b3-4bd5-92d9-605bd153a536 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:07:45.682117 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4dd3555-17b3-4bd5-92d9-605bd153a536 service neutron] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:07:45.714434 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 841/1648] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 10 12:07:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:07:45.813578 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4dd3555-17b3-4bd5-92d9-605bd153a536 service neutron] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:07:45.827287 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 808/1649] 10.222.0.46 () {60 vars in 1144 bytes} [Mon Oct 10 12:07:45 2022] POST /identity/v3/users => generated 282 bytes in 368 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 12:07:45.846570 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4dd3555-17b3-4bd5-92d9-605bd153a536 service neutron] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:07:45.880933 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd719d9b-ea68-4b26-b701-55ef06709900 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:07:45.886581 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd719d9b-ea68-4b26-b701-55ef06709900 admin admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:07:45.887297 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd719d9b-ea68-4b26-b701-55ef06709900 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:07:45.887583 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd719d9b-ea68-4b26-b701-55ef06709900 admin admin] PATH_INFO: `/v3/roles` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:07:45.888260 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd719d9b-ea68-4b26-b701-55ef06709900 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:07:45.889922 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd719d9b-ea68-4b26-b701-55ef06709900 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:07:45.931616 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 842/1650] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 10 12:07:45 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 12:07:45.934025 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9ba7e1-4521-4b6d-a50d-cd69de85ac80 admin admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:07:45.951214 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9ba7e1-4521-4b6d-a50d-cd69de85ac80 admin admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:07:45.963609 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fcf1010-67ce-4631-8989-babe8bed9e0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:07:45.964619 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcf1010-67ce-4631-8989-babe8bed9e0c admin admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:07:45.965276 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcf1010-67ce-4631-8989-babe8bed9e0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:07:45.965961 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcf1010-67ce-4631-8989-babe8bed9e0c admin admin] PATH_INFO: `/v3/roles` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:07:45.966910 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fcf1010-67ce-4631-8989-babe8bed9e0c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:07:45.968973 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fcf1010-67ce-4631-8989-babe8bed9e0c admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:07:45.990144 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd719d9b-ea68-4b26-b701-55ef06709900 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:07:46.008915 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 809/1651] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 10 12:07:45 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 12:07:46.028236 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd719d9b-ea68-4b26-b701-55ef06709900 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:07:46.050304 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aba7be9-9ddf-4384-b57e-0dafd311805f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:07:46.051044 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aba7be9-9ddf-4384-b57e-0dafd311805f admin admin] REQUEST_METHOD: `PUT` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:07:46.053918 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aba7be9-9ddf-4384-b57e-0dafd311805f admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:07:46.053918 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aba7be9-9ddf-4384-b57e-0dafd311805f admin admin] PATH_INFO: `/v3/projects/90cfbef072a2472a9049df9dcea4964f/users/9c7300856f374d35ba40cb7005dae437/roles/b2cb2a6e818e47338ce071e6fedfc673` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:07:46.118966 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aba7be9-9ddf-4384-b57e-0dafd311805f admin admin] RBAC: Authorizing `identity:create_grant(project_id=90cfbef072a2472a9049df9dcea4964f, user_id=9c7300856f374d35ba40cb7005dae437, role_id=b2cb2a6e818e47338ce071e6fedfc673)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:07:46.126765 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aba7be9-9ddf-4384-b57e-0dafd311805f admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:07:46.237066 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f69bdfc-5ef8-48b0-ae35-f7512e720953 None None] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:07:46.237419 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f69bdfc-5ef8-48b0-ae35-f7512e720953 None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:07:46.237697 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f69bdfc-5ef8-48b0-ae35-f7512e720953 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:07:46.248024 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 843/1652] 10.222.0.46 () {60 vars in 1483 bytes} [Mon Oct 10 12:07:45 2022] PUT /identity/v3/projects/90cfbef072a2472a9049df9dcea4964f/users/9c7300856f374d35ba40cb7005dae437/roles/b2cb2a6e818e47338ce071e6fedfc673 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:07:46.287653 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.auth.core [None req-4f69bdfc-5ef8-48b0-ae35-f7512e720953 None None] MFA Rules not processed for user `9c7300856f374d35ba40cb7005dae437`. Rule list: `[]` (Enabled: `True`). {{(pid=81676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:07:46.290962 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-4f69bdfc-5ef8-48b0-ae35-f7512e720953 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:07:46.523488 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 810/1653] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:07:46 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 288 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:07:46.528660 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aba7be9-9ddf-4384-b57e-0dafd311805f admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:07:46.704985 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aba7be9-9ddf-4384-b57e-0dafd311805f admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:07:46.719602 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54be062-3a5a-4023-addd-d09b2e21fa46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:07:46.720253 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54be062-3a5a-4023-addd-d09b2e21fa46 admin admin] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:07:46.720539 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54be062-3a5a-4023-addd-d09b2e21fa46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:07:46.720814 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54be062-3a5a-4023-addd-d09b2e21fa46 admin admin] PATH_INFO: `/v3/projects` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:07:46.732479 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54be062-3a5a-4023-addd-d09b2e21fa46 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:07:46.749792 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54be062-3a5a-4023-addd-d09b2e21fa46 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:07:46.920848 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 844/1654] 10.222.0.46 () {60 vars in 1153 bytes} [Mon Oct 10 12:07:46 2022] POST /identity/v3/projects => generated 365 bytes in 392 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 12:07:46.924500 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f69bdfc-5ef8-48b0-ae35-f7512e720953 None None] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:07:46.943759 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f69bdfc-5ef8-48b0-ae35-f7512e720953 None None] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:07:47.063430 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e366cda8-7555-447d-aa14-9d9a31e1faca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:07:47.064148 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e366cda8-7555-447d-aa14-9d9a31e1faca admin admin] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:07:47.064464 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e366cda8-7555-447d-aa14-9d9a31e1faca admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:07:47.064740 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e366cda8-7555-447d-aa14-9d9a31e1faca admin admin] PATH_INFO: `/v3/users` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:07:47.066121 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e366cda8-7555-447d-aa14-9d9a31e1faca admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:07:47.070089 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e366cda8-7555-447d-aa14-9d9a31e1faca admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:07:47.083240 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:07:47.167262 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 811/1655] 10.222.0.46 () {60 vars in 1144 bytes} [Mon Oct 10 12:07:46 2022] POST /identity/v3/users => generated 289 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 12:07:47.170710 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54be062-3a5a-4023-addd-d09b2e21fa46 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:07:47.186468 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54be062-3a5a-4023-addd-d09b2e21fa46 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:07:47.203711 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f319bd0-a069-41ee-a5c1-f066737ac63c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:07:47.205000 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f319bd0-a069-41ee-a5c1-f066737ac63c admin admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:07:47.205440 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f319bd0-a069-41ee-a5c1-f066737ac63c admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:07:47.205931 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f319bd0-a069-41ee-a5c1-f066737ac63c admin admin] PATH_INFO: `/v3/roles` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:07:47.206799 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f319bd0-a069-41ee-a5c1-f066737ac63c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:07:47.208691 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f319bd0-a069-41ee-a5c1-f066737ac63c admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:07:47.261603 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e366cda8-7555-447d-aa14-9d9a31e1faca admin admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:07:47.264041 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-e366cda8-7555-447d-aa14-9d9a31e1faca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:07:47.266071 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 845/1656] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 10 12:07:47 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 12:07:47.270578 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f319bd0-a069-41ee-a5c1-f066737ac63c admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:07:47.301063 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e366cda8-7555-447d-aa14-9d9a31e1faca admin admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:07:47.301584 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f319bd0-a069-41ee-a5c1-f066737ac63c admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:07:47.309316 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78f623e-b76c-45f3-aa01-9827db0bbdd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:07:47.309991 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78f623e-b76c-45f3-aa01-9827db0bbdd5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:07:47.310276 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78f623e-b76c-45f3-aa01-9827db0bbdd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:07:47.310557 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78f623e-b76c-45f3-aa01-9827db0bbdd5 admin admin] PATH_INFO: `/v3/users/df79e8908130422489d1354663a59b74` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:07:47.318047 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78f623e-b76c-45f3-aa01-9827db0bbdd5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=df79e8908130422489d1354663a59b74)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:07:47.324114 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86de8b63-65de-4195-ab41-c4ab87381bda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:07:47.324393 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78f623e-b76c-45f3-aa01-9827db0bbdd5 admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:07:47.329655 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86de8b63-65de-4195-ab41-c4ab87381bda admin admin] REQUEST_METHOD: `PUT` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:07:47.329655 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86de8b63-65de-4195-ab41-c4ab87381bda admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:07:47.329655 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86de8b63-65de-4195-ab41-c4ab87381bda admin admin] PATH_INFO: `/v3/projects/e67d2ed5531c4042bd67bf241e5ccea4/users/73d94fcd8e614f7699a61177362c18cf/roles/0d822a0674b6495fac623877a3b38025` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:07:47.362746 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86de8b63-65de-4195-ab41-c4ab87381bda admin admin] RBAC: Authorizing `identity:create_grant(project_id=e67d2ed5531c4042bd67bf241e5ccea4, user_id=73d94fcd8e614f7699a61177362c18cf, role_id=0d822a0674b6495fac623877a3b38025)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:07:47.365829 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86de8b63-65de-4195-ab41-c4ab87381bda admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:07:47.470637 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 846/1657] 10.222.0.46 () {60 vars in 1483 bytes} [Mon Oct 10 12:07:47 2022] PUT /identity/v3/projects/e67d2ed5531c4042bd67bf241e5ccea4/users/73d94fcd8e614f7699a61177362c18cf/roles/0d822a0674b6495fac623877a3b38025 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:07:47.474577 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86de8b63-65de-4195-ab41-c4ab87381bda admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:07:47.520836 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:07:47.638911 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86de8b63-65de-4195-ab41-c4ab87381bda admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:07:47.640221 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-d78f623e-b76c-45f3-aa01-9827db0bbdd5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'df79e8908130422489d1354663a59b74'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:07:47.656457 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a778873-81c9-430c-8062-90839b240242 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:07:47.658602 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a778873-81c9-430c-8062-90839b240242 admin admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:07:47.660407 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a778873-81c9-430c-8062-90839b240242 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:07:47.660769 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a778873-81c9-430c-8062-90839b240242 admin admin] PATH_INFO: `/v3/roles` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:07:47.667970 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-d78f623e-b76c-45f3-aa01-9827db0bbdd5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'df79e8908130422489d1354663a59b74'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:07:47.668442 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a778873-81c9-430c-8062-90839b240242 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:07:47.684444 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-d78f623e-b76c-45f3-aa01-9827db0bbdd5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'df79e8908130422489d1354663a59b74'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:07:47.685583 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a778873-81c9-430c-8062-90839b240242 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:07:47.719397 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 847/1658] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 10 12:07:47 2022] GET /identity/v3/roles => generated 1420 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 12:07:47.720465 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a778873-81c9-430c-8062-90839b240242 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:07:47.722395 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-3a778873-81c9-430c-8062-90839b240242 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:07:47.773535 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a778873-81c9-430c-8062-90839b240242 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:07:47.792773 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a81301e-fcf9-47bd-b832-99ca3cd3ed67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:07:47.793425 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a81301e-fcf9-47bd-b832-99ca3cd3ed67 admin admin] REQUEST_METHOD: `PUT` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:07:47.802073 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a81301e-fcf9-47bd-b832-99ca3cd3ed67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:07:47.803980 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a81301e-fcf9-47bd-b832-99ca3cd3ed67 admin admin] PATH_INFO: `/v3/projects/e67d2ed5531c4042bd67bf241e5ccea4/users/73d94fcd8e614f7699a61177362c18cf/roles/0d822a0674b6495fac623877a3b38025` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:07:47.810438 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a81301e-fcf9-47bd-b832-99ca3cd3ed67 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e67d2ed5531c4042bd67bf241e5ccea4, user_id=73d94fcd8e614f7699a61177362c18cf, role_id=0d822a0674b6495fac623877a3b38025)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:07:47.813493 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a81301e-fcf9-47bd-b832-99ca3cd3ed67 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:07:47.817775 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-d78f623e-b76c-45f3-aa01-9827db0bbdd5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'df79e8908130422489d1354663a59b74'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:07:47.850929 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-d78f623e-b76c-45f3-aa01-9827db0bbdd5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'df79e8908130422489d1354663a59b74'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:07:47.864474 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 848/1659] 10.222.0.46 () {60 vars in 1483 bytes} [Mon Oct 10 12:07:47 2022] PUT /identity/v3/projects/e67d2ed5531c4042bd67bf241e5ccea4/users/73d94fcd8e614f7699a61177362c18cf/roles/0d822a0674b6495fac623877a3b38025 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:07:47.877878 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f62232c-9a0d-43c8-b991-455b9fd0857d None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:07:47.882392 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 812/1660] 10.222.0.46 () {58 vars in 1224 bytes} [Mon Oct 10 12:07:47 2022] DELETE /identity/v3/users/df79e8908130422489d1354663a59b74 => generated 0 bytes in 618 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 10 12:07:47.884667 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1111e202-ecac-4432-bb09-9c1c1829fdfb None None] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:07:47.885586 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f62232c-9a0d-43c8-b991-455b9fd0857d None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:07:47.885871 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f62232c-9a0d-43c8-b991-455b9fd0857d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:07:47.896508 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1111e202-ecac-4432-bb09-9c1c1829fdfb None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:07:47.896759 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1111e202-ecac-4432-bb09-9c1c1829fdfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:07:47.954304 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.auth.core [None req-1111e202-ecac-4432-bb09-9c1c1829fdfb None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:07:47.959491 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-1111e202-ecac-4432-bb09-9c1c1829fdfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:07:47.972277 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-3f62232c-9a0d-43c8-b991-455b9fd0857d None None] MFA Rules not processed for user `73d94fcd8e614f7699a61177362c18cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:07:47.975391 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-3f62232c-9a0d-43c8-b991-455b9fd0857d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:07:48.115418 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 813/1661] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:07:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:07:48.117881 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1111e202-ecac-4432-bb09-9c1c1829fdfb None None] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:07:48.121073 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-1111e202-ecac-4432-bb09-9c1c1829fdfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:07:48.151655 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1111e202-ecac-4432-bb09-9c1c1829fdfb None None] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:07:48.155623 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 849/1662] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:07:47 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 281 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 10 12:07:48.168141 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2f57d7-36a7-42c4-9b94-cd9113e615e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:07:48.168533 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f62232c-9a0d-43c8-b991-455b9fd0857d None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:07:48.168883 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2f57d7-36a7-42c4-9b94-cd9113e615e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:07:48.169169 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2f57d7-36a7-42c4-9b94-cd9113e615e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:07:48.169447 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2f57d7-36a7-42c4-9b94-cd9113e615e8 admin admin] PATH_INFO: `/v3/projects/b3c137fcc1464218b989ce343e84cd22` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:07:48.175544 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-3f62232c-9a0d-43c8-b991-455b9fd0857d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:07:48.176584 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2f57d7-36a7-42c4-9b94-cd9113e615e8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b3c137fcc1464218b989ce343e84cd22)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:07:48.178658 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2f57d7-36a7-42c4-9b94-cd9113e615e8 admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:07:48.293161 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f62232c-9a0d-43c8-b991-455b9fd0857d None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:07:48.297384 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c07d08e-64ee-438b-bf31-5e2465660bc0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3ae093470484b93b23dcfae1657df9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad3772f7b6a47c28b7ba37a9316a3c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:07:48.297878 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c07d08e-64ee-438b-bf31-5e2465660bc0 service cinder] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:07:48.298158 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c07d08e-64ee-438b-bf31-5e2465660bc0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:07:48.298483 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c07d08e-64ee-438b-bf31-5e2465660bc0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:07:48.309116 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c07d08e-64ee-438b-bf31-5e2465660bc0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:07:48.311067 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c07d08e-64ee-438b-bf31-5e2465660bc0 service cinder] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:07:48.344222 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 850/1663] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 10 12:07:48 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 10 12:07:48.348936 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c07d08e-64ee-438b-bf31-5e2465660bc0 service cinder] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:07:48.353026 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-9c07d08e-64ee-438b-bf31-5e2465660bc0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:07:48.363369 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-dd2f57d7-36a7-42c4-9b94-cd9113e615e8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b3c137fcc1464218b989ce343e84cd22', 'request_id': 'req-dd2f57d7-36a7-42c4-9b94-cd9113e615e8'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:07:48.379615 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-dd2f57d7-36a7-42c4-9b94-cd9113e615e8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b3c137fcc1464218b989ce343e84cd22', 'request_id': 'req-dd2f57d7-36a7-42c4-9b94-cd9113e615e8'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:07:48.390685 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-dd2f57d7-36a7-42c4-9b94-cd9113e615e8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b3c137fcc1464218b989ce343e84cd22', 'request_id': 'req-dd2f57d7-36a7-42c4-9b94-cd9113e615e8'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:07:48.395330 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:07:48.472180 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c07d08e-64ee-438b-bf31-5e2465660bc0 service cinder] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:07:48.510885 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-dd2f57d7-36a7-42c4-9b94-cd9113e615e8 admin admin] The token cache is being invalidate because project b3c137fcc1464218b989ce343e84cd22 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81676) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 10 12:07:48.511437 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-dd2f57d7-36a7-42c4-9b94-cd9113e615e8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:07:48.513257 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-dd2f57d7-36a7-42c4-9b94-cd9113e615e8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:07:48.516356 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 814/1664] 10.222.0.46 () {58 vars in 1233 bytes} [Mon Oct 10 12:07:48 2022] DELETE /identity/v3/projects/b3c137fcc1464218b989ce343e84cd22 => generated 0 bytes in 400 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:07:48.522429 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e0f618-9b92-4d6e-847a-6806b1e0dd51 None None] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:07:48.523726 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e0f618-9b92-4d6e-847a-6806b1e0dd51 None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:07:48.524870 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e0f618-9b92-4d6e-847a-6806b1e0dd51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:07:48.564078 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:07:48.580267 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.auth.core [None req-33e0f618-9b92-4d6e-847a-6806b1e0dd51 None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:07:48.581771 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-33e0f618-9b92-4d6e-847a-6806b1e0dd51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:07:48.607653 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b0beec-2cec-4cef-9405-98de3ecf316e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58b1c7c85302421cba823ace4201861f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad3772f7b6a47c28b7ba37a9316a3c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:07:48.607653 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b0beec-2cec-4cef-9405-98de3ecf316e service neutron] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:07:48.607653 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b0beec-2cec-4cef-9405-98de3ecf316e service neutron] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:07:48.609819 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b0beec-2cec-4cef-9405-98de3ecf316e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:07:48.614136 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-86b0beec-2cec-4cef-9405-98de3ecf316e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:07:48.715634 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 815/1665] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:07:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:07:48.717877 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e0f618-9b92-4d6e-847a-6806b1e0dd51 None None] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:07:48.720627 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-33e0f618-9b92-4d6e-847a-6806b1e0dd51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:07:48.753178 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b0beec-2cec-4cef-9405-98de3ecf316e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:07:48.755608 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b0beec-2cec-4cef-9405-98de3ecf316e service neutron] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:07:48.779931 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 851/1666] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 10 12:07:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 435 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:07:48.782157 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5141838-d2a6-4668-aab1-9494dc1432ec None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:07:48.782523 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5141838-d2a6-4668-aab1-9494dc1432ec None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:07:48.783713 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5141838-d2a6-4668-aab1-9494dc1432ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:07:48.826718 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-d5141838-d2a6-4668-aab1-9494dc1432ec None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:07:48.827098 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e0f618-9b92-4d6e-847a-6806b1e0dd51 None None] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:07:48.829445 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-d5141838-d2a6-4668-aab1-9494dc1432ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:07:48.836125 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77488eaa-53c3-4403-8958-567b7772b25b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3ae093470484b93b23dcfae1657df9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad3772f7b6a47c28b7ba37a9316a3c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:07:48.840535 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77488eaa-53c3-4403-8958-567b7772b25b service cinder] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:07:48.842001 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77488eaa-53c3-4403-8958-567b7772b25b service cinder] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:07:48.842404 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77488eaa-53c3-4403-8958-567b7772b25b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:07:48.845177 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-77488eaa-53c3-4403-8958-567b7772b25b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:07:48.854701 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 852/1667] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:07:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:07:48.860244 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5141838-d2a6-4668-aab1-9494dc1432ec None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:07:48.897917 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5141838-d2a6-4668-aab1-9494dc1432ec None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:07:48.906015 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d824e0a-91fd-47db-83d4-ed425fecec62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:07:48.909643 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d824e0a-91fd-47db-83d4-ed425fecec62 admin admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:07:48.911630 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d824e0a-91fd-47db-83d4-ed425fecec62 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:07:48.912105 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d824e0a-91fd-47db-83d4-ed425fecec62 admin admin] PATH_INFO: `/v3/domains` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:07:48.917139 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d824e0a-91fd-47db-83d4-ed425fecec62 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 12:07:48.918416 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d824e0a-91fd-47db-83d4-ed425fecec62 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:07:48.924110 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d824e0a-91fd-47db-83d4-ed425fecec62 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:07:48.946568 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 853/1668] 10.222.0.46 () {58 vars in 1153 bytes} [Mon Oct 10 12:07:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 12:07:48.947518 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77488eaa-53c3-4403-8958-567b7772b25b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:07:48.949025 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77488eaa-53c3-4403-8958-567b7772b25b service cinder] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:07:48.953596 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d824e0a-91fd-47db-83d4-ed425fecec62 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:07:48.987416 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 816/1669] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 10 12:07:48 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 270 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:07:48.990888 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d824e0a-91fd-47db-83d4-ed425fecec62 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:07:49.004256 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b81478-4138-48ad-bb37-d018cbf4a7c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:07:49.005793 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b81478-4138-48ad-bb37-d018cbf4a7c5 admin admin] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:07:49.005793 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b81478-4138-48ad-bb37-d018cbf4a7c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:07:49.005793 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b81478-4138-48ad-bb37-d018cbf4a7c5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:07:49.006090 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b81478-4138-48ad-bb37-d018cbf4a7c5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:07:49.009930 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b81478-4138-48ad-bb37-d018cbf4a7c5 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:07:49.185109 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 854/1670] 10.222.0.46 () {60 vars in 1153 bytes} [Mon Oct 10 12:07:48 2022] POST /identity/v3/projects => generated 403 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 12:07:49.187691 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77488eaa-53c3-4403-8958-567b7772b25b service cinder] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:07:49.208440 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77488eaa-53c3-4403-8958-567b7772b25b service cinder] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:07:49.369922 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:INSERT=1,keystone:DELETE=1,keystone:SELECT=98 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:07:49.393637 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bb40c7-a3f2-4725-8ce4-f65baecda73a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:07:49.396692 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bb40c7-a3f2-4725-8ce4-f65baecda73a admin admin] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:07:49.398651 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bb40c7-a3f2-4725-8ce4-f65baecda73a admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:07:49.399943 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bb40c7-a3f2-4725-8ce4-f65baecda73a admin admin] PATH_INFO: `/v3/users` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:07:49.402463 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28bb40c7-a3f2-4725-8ce4-f65baecda73a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:07:49.405708 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28bb40c7-a3f2-4725-8ce4-f65baecda73a admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:07:49.558492 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 817/1671] 10.222.0.46 () {60 vars in 1144 bytes} [Mon Oct 10 12:07:49 2022] POST /identity/v3/users => generated 302 bytes in 371 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 12:07:49.563825 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b81478-4138-48ad-bb37-d018cbf4a7c5 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:07:49.581021 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b81478-4138-48ad-bb37-d018cbf4a7c5 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:07:49.592781 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c862b99-7243-412e-88cd-9f92886a4eb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:07:49.593776 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c862b99-7243-412e-88cd-9f92886a4eb9 admin admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:07:49.595685 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c862b99-7243-412e-88cd-9f92886a4eb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:07:49.596180 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c862b99-7243-412e-88cd-9f92886a4eb9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:07:49.597269 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c862b99-7243-412e-88cd-9f92886a4eb9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:07:49.599610 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c862b99-7243-412e-88cd-9f92886a4eb9 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:07:49.646986 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 855/1672] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 10 12:07:49 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 10 12:07:49.652859 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bb40c7-a3f2-4725-8ce4-f65baecda73a admin admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:07:49.688333 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bb40c7-a3f2-4725-8ce4-f65baecda73a admin admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:07:49.708549 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa11bf2-c044-4fd8-b601-c0fcc84a0664 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:07:49.710251 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa11bf2-c044-4fd8-b601-c0fcc84a0664 admin admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:07:49.710646 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa11bf2-c044-4fd8-b601-c0fcc84a0664 admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:07:49.710888 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa11bf2-c044-4fd8-b601-c0fcc84a0664 admin admin] PATH_INFO: `/v3/roles` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:07:49.714367 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa11bf2-c044-4fd8-b601-c0fcc84a0664 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:07:49.719167 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa11bf2-c044-4fd8-b601-c0fcc84a0664 admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:07:49.759257 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 818/1673] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 10 12:07:49 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 12:07:49.763568 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c862b99-7243-412e-88cd-9f92886a4eb9 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:07:49.779877 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c862b99-7243-412e-88cd-9f92886a4eb9 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:07:49.796572 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff737428-bbe4-4a83-ab94-3a7740f44c5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:07:49.797401 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff737428-bbe4-4a83-ab94-3a7740f44c5a admin admin] REQUEST_METHOD: `PUT` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:07:49.797695 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff737428-bbe4-4a83-ab94-3a7740f44c5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:07:49.798134 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff737428-bbe4-4a83-ab94-3a7740f44c5a admin admin] PATH_INFO: `/v3/projects/b8bbbb12912d456c82b35bac26a7d1a9/users/07fdf81df3714bdf8fbdd8e999136223/roles/b2cb2a6e818e47338ce071e6fedfc673` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:07:49.838833 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff737428-bbe4-4a83-ab94-3a7740f44c5a admin admin] RBAC: Authorizing `identity:create_grant(project_id=b8bbbb12912d456c82b35bac26a7d1a9, user_id=07fdf81df3714bdf8fbdd8e999136223, role_id=b2cb2a6e818e47338ce071e6fedfc673)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:07:49.842181 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff737428-bbe4-4a83-ab94-3a7740f44c5a admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:07:49.948330 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 856/1674] 10.222.0.46 () {60 vars in 1483 bytes} [Mon Oct 10 12:07:49 2022] PUT /identity/v3/projects/b8bbbb12912d456c82b35bac26a7d1a9/users/07fdf81df3714bdf8fbdd8e999136223/roles/b2cb2a6e818e47338ce071e6fedfc673 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:07:49.970979 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca9e643-ae49-4a2e-a8ac-e44bedd8a7d1 None None] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:07:49.971461 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca9e643-ae49-4a2e-a8ac-e44bedd8a7d1 None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:07:49.971768 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca9e643-ae49-4a2e-a8ac-e44bedd8a7d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:07:50.020890 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.auth.core [None req-aca9e643-ae49-4a2e-a8ac-e44bedd8a7d1 None None] MFA Rules not processed for user `07fdf81df3714bdf8fbdd8e999136223`. Rule list: `[]` (Enabled: `True`). {{(pid=81676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:07:50.022344 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-aca9e643-ae49-4a2e-a8ac-e44bedd8a7d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:07:50.206442 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 819/1675] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:07:49 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:07:50.210659 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff737428-bbe4-4a83-ab94-3a7740f44c5a admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:07:50.297006 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:07:50.436381 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff737428-bbe4-4a83-ab94-3a7740f44c5a admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:07:50.456001 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4daad48b-487c-47aa-90fa-71e59bc47769 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:07:50.457350 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4daad48b-487c-47aa-90fa-71e59bc47769 admin admin] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:07:50.457864 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4daad48b-487c-47aa-90fa-71e59bc47769 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:07:50.458352 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4daad48b-487c-47aa-90fa-71e59bc47769 admin admin] PATH_INFO: `/v3/projects` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:07:50.467320 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4daad48b-487c-47aa-90fa-71e59bc47769 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:07:50.469724 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4daad48b-487c-47aa-90fa-71e59bc47769 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:07:50.622356 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 857/1676] 10.222.0.46 () {60 vars in 1153 bytes} [Mon Oct 10 12:07:50 2022] POST /identity/v3/projects => generated 405 bytes in 409 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 12:07:50.623748 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca9e643-ae49-4a2e-a8ac-e44bedd8a7d1 None None] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:07:50.643086 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca9e643-ae49-4a2e-a8ac-e44bedd8a7d1 None None] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:07:50.779602 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30bebd6-1c20-44e1-8cf7-454dcc3fcde4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:07:50.780248 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30bebd6-1c20-44e1-8cf7-454dcc3fcde4 admin admin] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:07:50.780526 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30bebd6-1c20-44e1-8cf7-454dcc3fcde4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:07:50.780798 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30bebd6-1c20-44e1-8cf7-454dcc3fcde4 admin admin] PATH_INFO: `/v3/users` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:07:50.781665 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30bebd6-1c20-44e1-8cf7-454dcc3fcde4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:07:50.788055 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30bebd6-1c20-44e1-8cf7-454dcc3fcde4 admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:07:50.836044 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4daad48b-487c-47aa-90fa-71e59bc47769 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:07:50.838896 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-4daad48b-487c-47aa-90fa-71e59bc47769 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:07:50.860935 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4daad48b-487c-47aa-90fa-71e59bc47769 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:07:50.873081 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1ecb0c-931a-4b30-ad45-9559e65da812 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:07:50.873754 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1ecb0c-931a-4b30-ad45-9559e65da812 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:07:50.874032 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1ecb0c-931a-4b30-ad45-9559e65da812 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:07:50.874491 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1ecb0c-931a-4b30-ad45-9559e65da812 admin admin] PATH_INFO: `/v3/projects/65f70a82ea624589925400272b757dea` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:07:50.877981 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1ecb0c-931a-4b30-ad45-9559e65da812 admin admin] RBAC: Authorizing `identity:delete_project(project_id=65f70a82ea624589925400272b757dea)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:07:50.880524 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1ecb0c-931a-4b30-ad45-9559e65da812 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:07:50.949539 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 820/1677] 10.222.0.46 () {60 vars in 1144 bytes} [Mon Oct 10 12:07:50 2022] POST /identity/v3/users => generated 309 bytes in 326 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 12:07:50.957320 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30bebd6-1c20-44e1-8cf7-454dcc3fcde4 admin admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:07:50.985247 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30bebd6-1c20-44e1-8cf7-454dcc3fcde4 admin admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:07:51.003262 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e859bda-9ddb-4e0c-970c-16a16fbfc11e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:07:51.006030 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e859bda-9ddb-4e0c-970c-16a16fbfc11e admin admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:07:51.006316 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e859bda-9ddb-4e0c-970c-16a16fbfc11e admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:07:51.006588 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e859bda-9ddb-4e0c-970c-16a16fbfc11e admin admin] PATH_INFO: `/v3/roles` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:07:51.007956 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e859bda-9ddb-4e0c-970c-16a16fbfc11e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:07:51.009554 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e859bda-9ddb-4e0c-970c-16a16fbfc11e admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:07:51.051042 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 821/1678] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 10 12:07:50 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 12:07:51.055870 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e859bda-9ddb-4e0c-970c-16a16fbfc11e admin admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:07:51.071074 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e859bda-9ddb-4e0c-970c-16a16fbfc11e admin admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:07:51.100303 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-dc1ecb0c-931a-4b30-ad45-9559e65da812 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '65f70a82ea624589925400272b757dea', 'request_id': 'req-dc1ecb0c-931a-4b30-ad45-9559e65da812'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:07:51.111443 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:07:51.117010 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-dc1ecb0c-931a-4b30-ad45-9559e65da812 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '65f70a82ea624589925400272b757dea', 'request_id': 'req-dc1ecb0c-931a-4b30-ad45-9559e65da812'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:07:51.139810 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-dc1ecb0c-931a-4b30-ad45-9559e65da812 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '65f70a82ea624589925400272b757dea', 'request_id': 'req-dc1ecb0c-931a-4b30-ad45-9559e65da812'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:07:51.152412 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a997ea-ddcb-4ac8-a2a6-1c6e09583394 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:07:51.153161 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a997ea-ddcb-4ac8-a2a6-1c6e09583394 admin admin] REQUEST_METHOD: `PUT` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:07:51.154030 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a997ea-ddcb-4ac8-a2a6-1c6e09583394 admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:07:51.154314 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a997ea-ddcb-4ac8-a2a6-1c6e09583394 admin admin] PATH_INFO: `/v3/projects/67dbb5f9c61a4cf0883d7f5376f3a0af/users/cbd1e94ae9a44de3bf190ef8959961b4/roles/0d822a0674b6495fac623877a3b38025` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:07:51.173973 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a997ea-ddcb-4ac8-a2a6-1c6e09583394 admin admin] RBAC: Authorizing `identity:create_grant(project_id=67dbb5f9c61a4cf0883d7f5376f3a0af, user_id=cbd1e94ae9a44de3bf190ef8959961b4, role_id=0d822a0674b6495fac623877a3b38025)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:07:51.176924 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a997ea-ddcb-4ac8-a2a6-1c6e09583394 admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:07:51.249916 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-dc1ecb0c-931a-4b30-ad45-9559e65da812 admin admin] The token cache is being invalidate because project 65f70a82ea624589925400272b757dea was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81677) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 10 12:07:51.251019 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-dc1ecb0c-931a-4b30-ad45-9559e65da812 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:07:51.251335 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-dc1ecb0c-931a-4b30-ad45-9559e65da812 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:07:51.258511 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 858/1679] 10.222.0.46 () {58 vars in 1233 bytes} [Mon Oct 10 12:07:50 2022] DELETE /identity/v3/projects/65f70a82ea624589925400272b757dea => generated 0 bytes in 423 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:07:51.262521 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1ecb0c-931a-4b30-ad45-9559e65da812 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:07:51.267296 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-dc1ecb0c-931a-4b30-ad45-9559e65da812 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:07:51.290798 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 822/1680] 10.222.0.46 () {60 vars in 1483 bytes} [Mon Oct 10 12:07:51 2022] PUT /identity/v3/projects/67dbb5f9c61a4cf0883d7f5376f3a0af/users/cbd1e94ae9a44de3bf190ef8959961b4/roles/0d822a0674b6495fac623877a3b38025 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:07:51.294867 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a997ea-ddcb-4ac8-a2a6-1c6e09583394 admin admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:07:51.297189 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1ecb0c-931a-4b30-ad45-9559e65da812 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:07:51.298521 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-58a997ea-ddcb-4ac8-a2a6-1c6e09583394 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:07:51.431001 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4b1885-4cc7-4b0c-a7df-32a91c57be31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:07:51.431671 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4b1885-4cc7-4b0c-a7df-32a91c57be31 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:07:51.431959 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4b1885-4cc7-4b0c-a7df-32a91c57be31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:07:51.432234 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4b1885-4cc7-4b0c-a7df-32a91c57be31 admin admin] PATH_INFO: `/v3/users/a8ddba03273842d3b41ef01cd904e0b1` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:07:51.433434 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4b1885-4cc7-4b0c-a7df-32a91c57be31 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a8ddba03273842d3b41ef01cd904e0b1)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:07:51.435452 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4b1885-4cc7-4b0c-a7df-32a91c57be31 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:07:51.484147 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a997ea-ddcb-4ac8-a2a6-1c6e09583394 admin admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:07:51.490184 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca899a4-94e0-42c9-a1d6-c872d0d49c1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:07:51.490981 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca899a4-94e0-42c9-a1d6-c872d0d49c1a admin admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:07:51.491386 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca899a4-94e0-42c9-a1d6-c872d0d49c1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:07:51.492277 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca899a4-94e0-42c9-a1d6-c872d0d49c1a admin admin] PATH_INFO: `/v3/roles` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:07:51.496006 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca899a4-94e0-42c9-a1d6-c872d0d49c1a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:07:51.501795 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca899a4-94e0-42c9-a1d6-c872d0d49c1a admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:07:51.523000 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 823/1681] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 10 12:07:51 2022] GET /identity/v3/roles => generated 1420 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 10 12:07:51.528455 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca899a4-94e0-42c9-a1d6-c872d0d49c1a admin admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:07:51.551626 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca899a4-94e0-42c9-a1d6-c872d0d49c1a admin admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:07:51.559608 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=86,keystone:INSERT=2,keystone:DELETE=12 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:07:51.565940 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc62419-916e-4749-9cb8-193f09c019a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:07:51.568165 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc62419-916e-4749-9cb8-193f09c019a7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:07:51.569487 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc62419-916e-4749-9cb8-193f09c019a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:07:51.571331 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc62419-916e-4749-9cb8-193f09c019a7 admin admin] PATH_INFO: `/v3/projects/67dbb5f9c61a4cf0883d7f5376f3a0af/users/cbd1e94ae9a44de3bf190ef8959961b4/roles/0d822a0674b6495fac623877a3b38025` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:07:51.582043 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc62419-916e-4749-9cb8-193f09c019a7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=67dbb5f9c61a4cf0883d7f5376f3a0af, user_id=cbd1e94ae9a44de3bf190ef8959961b4, role_id=0d822a0674b6495fac623877a3b38025)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:07:51.585001 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc62419-916e-4749-9cb8-193f09c019a7 admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:07:51.612205 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 824/1682] 10.222.0.46 () {60 vars in 1483 bytes} [Mon Oct 10 12:07:51 2022] PUT /identity/v3/projects/67dbb5f9c61a4cf0883d7f5376f3a0af/users/cbd1e94ae9a44de3bf190ef8959961b4/roles/0d822a0674b6495fac623877a3b38025 => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:07:51.623053 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ddce50-afb9-42f4-99ad-8fc845c2799b None None] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:07:51.623348 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ddce50-afb9-42f4-99ad-8fc845c2799b None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:07:51.623629 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ddce50-afb9-42f4-99ad-8fc845c2799b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:07:51.687062 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-de4b1885-4cc7-4b0c-a7df-32a91c57be31 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a8ddba03273842d3b41ef01cd904e0b1'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:07:51.687286 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.auth.core [None req-b7ddce50-afb9-42f4-99ad-8fc845c2799b None None] MFA Rules not processed for user `cbd1e94ae9a44de3bf190ef8959961b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:07:51.688551 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-b7ddce50-afb9-42f4-99ad-8fc845c2799b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:07:51.702814 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-de4b1885-4cc7-4b0c-a7df-32a91c57be31 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a8ddba03273842d3b41ef01cd904e0b1'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:07:51.708675 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-de4b1885-4cc7-4b0c-a7df-32a91c57be31 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a8ddba03273842d3b41ef01cd904e0b1'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:07:51.814599 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-de4b1885-4cc7-4b0c-a7df-32a91c57be31 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a8ddba03273842d3b41ef01cd904e0b1'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:07:51.853073 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 825/1683] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:07:51 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:07:51.862276 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ddce50-afb9-42f4-99ad-8fc845c2799b None None] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:07:51.863747 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-b7ddce50-afb9-42f4-99ad-8fc845c2799b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:07:51.870121 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-de4b1885-4cc7-4b0c-a7df-32a91c57be31 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a8ddba03273842d3b41ef01cd904e0b1'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:07:51.891114 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 859/1684] 10.222.0.46 () {58 vars in 1224 bytes} [Mon Oct 10 12:07:51 2022] DELETE /identity/v3/users/a8ddba03273842d3b41ef01cd904e0b1 => generated 0 bytes in 629 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 10 12:07:51.929173 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:07:52.005271 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ddce50-afb9-42f4-99ad-8fc845c2799b None None] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:07:52.131738 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efd3b86-9dc8-473f-9c9a-eb02825be2a2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3ae093470484b93b23dcfae1657df9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad3772f7b6a47c28b7ba37a9316a3c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:07:52.136745 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efd3b86-9dc8-473f-9c9a-eb02825be2a2 service cinder] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:07:52.136745 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efd3b86-9dc8-473f-9c9a-eb02825be2a2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:07:52.136745 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efd3b86-9dc8-473f-9c9a-eb02825be2a2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:07:52.247704 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8efd3b86-9dc8-473f-9c9a-eb02825be2a2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:07:52.249972 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8efd3b86-9dc8-473f-9c9a-eb02825be2a2 service cinder] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:07:52.266040 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 826/1685] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 10 12:07:51 2022] GET /identity/v3/auth/tokens => generated 3349 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 10 12:07:52.541588 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4b1885-4cc7-4b0c-a7df-32a91c57be31 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:07:52.543808 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-de4b1885-4cc7-4b0c-a7df-32a91c57be31 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:07:52.780471 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4b1885-4cc7-4b0c-a7df-32a91c57be31 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:07:52.802143 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259310bf-1b42-417a-b7f6-b8d679f5bea0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:07:52.803013 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259310bf-1b42-417a-b7f6-b8d679f5bea0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:07:52.807103 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259310bf-1b42-417a-b7f6-b8d679f5bea0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:07:52.807636 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259310bf-1b42-417a-b7f6-b8d679f5bea0 admin admin] PATH_INFO: `/v3/users/07fdf81df3714bdf8fbdd8e999136223` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:07:52.811478 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259310bf-1b42-417a-b7f6-b8d679f5bea0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=07fdf81df3714bdf8fbdd8e999136223)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:07:52.819457 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259310bf-1b42-417a-b7f6-b8d679f5bea0 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:07:53.089634 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-259310bf-1b42-417a-b7f6-b8d679f5bea0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '07fdf81df3714bdf8fbdd8e999136223'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:07:53.102443 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-259310bf-1b42-417a-b7f6-b8d679f5bea0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '07fdf81df3714bdf8fbdd8e999136223'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:07:53.114032 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-259310bf-1b42-417a-b7f6-b8d679f5bea0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '07fdf81df3714bdf8fbdd8e999136223'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:07:53.218102 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-259310bf-1b42-417a-b7f6-b8d679f5bea0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '07fdf81df3714bdf8fbdd8e999136223'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:07:53.259766 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-259310bf-1b42-417a-b7f6-b8d679f5bea0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '07fdf81df3714bdf8fbdd8e999136223'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:07:53.281262 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 860/1686] 10.222.0.46 () {58 vars in 1224 bytes} [Mon Oct 10 12:07:52 2022] DELETE /identity/v3/users/07fdf81df3714bdf8fbdd8e999136223 => generated 0 bytes in 740 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:07:53.288284 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f9ffce-3c52-4f45-91c1-09257c8576ce None None] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:07:53.289677 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f9ffce-3c52-4f45-91c1-09257c8576ce None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:07:53.290091 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f9ffce-3c52-4f45-91c1-09257c8576ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:07:53.340875 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.auth.core [None req-35f9ffce-3c52-4f45-91c1-09257c8576ce None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:07:53.344672 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-35f9ffce-3c52-4f45-91c1-09257c8576ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:07:53.534829 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 827/1687] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:07:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:07:53.553795 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259310bf-1b42-417a-b7f6-b8d679f5bea0 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:07:53.556427 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-259310bf-1b42-417a-b7f6-b8d679f5bea0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:07:53.623989 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:07:53.680834 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259310bf-1b42-417a-b7f6-b8d679f5bea0 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:07:53.686546 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4632a8b6-861f-4ad1-8951-da6690729c2b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58b1c7c85302421cba823ace4201861f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad3772f7b6a47c28b7ba37a9316a3c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:07:53.687499 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4632a8b6-861f-4ad1-8951-da6690729c2b service neutron] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:07:53.688155 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4632a8b6-861f-4ad1-8951-da6690729c2b service neutron] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:07:53.688637 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4632a8b6-861f-4ad1-8951-da6690729c2b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:07:53.703735 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4632a8b6-861f-4ad1-8951-da6690729c2b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:07:53.705593 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4632a8b6-861f-4ad1-8951-da6690729c2b service neutron] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:07:53.717884 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 861/1688] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 10 12:07:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:07:54.758470 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f9ffce-3c52-4f45-91c1-09257c8576ce None None] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:07:54.764302 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-35f9ffce-3c52-4f45-91c1-09257c8576ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:07:54.797307 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f9ffce-3c52-4f45-91c1-09257c8576ce None None] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:07:54.809972 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c46279c-0688-40a0-adaa-5f2773a8f66e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:07:54.815579 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c46279c-0688-40a0-adaa-5f2773a8f66e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:07:54.815579 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c46279c-0688-40a0-adaa-5f2773a8f66e admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:07:54.815579 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c46279c-0688-40a0-adaa-5f2773a8f66e admin admin] PATH_INFO: `/v3/projects/abfd589aa65542ca96d05012a56664a8` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:07:54.815579 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c46279c-0688-40a0-adaa-5f2773a8f66e admin admin] RBAC: Authorizing `identity:delete_project(project_id=abfd589aa65542ca96d05012a56664a8)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:07:54.820509 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c46279c-0688-40a0-adaa-5f2773a8f66e admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:07:54.969246 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:07:55.000596 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-6c46279c-0688-40a0-adaa-5f2773a8f66e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'abfd589aa65542ca96d05012a56664a8', 'request_id': 'req-6c46279c-0688-40a0-adaa-5f2773a8f66e'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:07:55.012887 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-6c46279c-0688-40a0-adaa-5f2773a8f66e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'abfd589aa65542ca96d05012a56664a8', 'request_id': 'req-6c46279c-0688-40a0-adaa-5f2773a8f66e'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:07:55.027958 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-6c46279c-0688-40a0-adaa-5f2773a8f66e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'abfd589aa65542ca96d05012a56664a8', 'request_id': 'req-6c46279c-0688-40a0-adaa-5f2773a8f66e'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:07:55.166194 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-6c46279c-0688-40a0-adaa-5f2773a8f66e admin admin] The token cache is being invalidate because project abfd589aa65542ca96d05012a56664a8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81676) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 10 12:07:55.166194 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-6c46279c-0688-40a0-adaa-5f2773a8f66e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:07:55.166194 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-6c46279c-0688-40a0-adaa-5f2773a8f66e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:07:55.167017 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 828/1689] 10.222.0.46 () {58 vars in 1233 bytes} [Mon Oct 10 12:07:54 2022] DELETE /identity/v3/projects/abfd589aa65542ca96d05012a56664a8 => generated 0 bytes in 411 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:07:55.172097 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c250aae-6f36-4ce4-825e-07cf2b02ba4e None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:07:55.175336 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c250aae-6f36-4ce4-825e-07cf2b02ba4e None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:07:55.176962 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c250aae-6f36-4ce4-825e-07cf2b02ba4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:07:55.230810 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-1c250aae-6f36-4ce4-825e-07cf2b02ba4e None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:07:55.233768 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-1c250aae-6f36-4ce4-825e-07cf2b02ba4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:07:55.436580 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 862/1690] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:07:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 257 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:07:55.437802 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dce4f3a-4fa0-4c1f-9903-9bfd4c1bf0e8 None None] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:07:55.438096 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dce4f3a-4fa0-4c1f-9903-9bfd4c1bf0e8 None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:07:55.438507 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dce4f3a-4fa0-4c1f-9903-9bfd4c1bf0e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:07:55.478281 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.auth.core [None req-8dce4f3a-4fa0-4c1f-9903-9bfd4c1bf0e8 None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:07:55.480255 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-8dce4f3a-4fa0-4c1f-9903-9bfd4c1bf0e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:07:55.501751 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 829/1691] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:07:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:07:55.502621 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c250aae-6f36-4ce4-825e-07cf2b02ba4e None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:07:55.514041 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c250aae-6f36-4ce4-825e-07cf2b02ba4e None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:07:55.541848 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4677983f-606d-4590-82a7-869f2c7367df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:07:55.544716 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4677983f-606d-4590-82a7-869f2c7367df admin admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:07:55.544716 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4677983f-606d-4590-82a7-869f2c7367df admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:07:55.544716 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4677983f-606d-4590-82a7-869f2c7367df admin admin] PATH_INFO: `/v3/domains` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:07:55.544716 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4677983f-606d-4590-82a7-869f2c7367df admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 12:07:55.544716 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4677983f-606d-4590-82a7-869f2c7367df admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:07:55.545610 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4677983f-606d-4590-82a7-869f2c7367df admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:07:55.580623 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 863/1692] 10.222.0.46 () {58 vars in 1153 bytes} [Mon Oct 10 12:07:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 12:07:55.582278 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dce4f3a-4fa0-4c1f-9903-9bfd4c1bf0e8 None None] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:07:55.603186 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dce4f3a-4fa0-4c1f-9903-9bfd4c1bf0e8 None None] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:07:55.611926 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa26d093-7a0e-4979-9e15-baecef2af796 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:07:55.612778 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa26d093-7a0e-4979-9e15-baecef2af796 admin admin] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:07:55.613585 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa26d093-7a0e-4979-9e15-baecef2af796 admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:07:55.614468 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa26d093-7a0e-4979-9e15-baecef2af796 admin admin] PATH_INFO: `/v3/projects` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:07:55.615392 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa26d093-7a0e-4979-9e15-baecef2af796 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:07:55.621749 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa26d093-7a0e-4979-9e15-baecef2af796 admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:07:55.633918 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6d2a4a-99f1-4280-8eeb-10c9bb5a6ba5 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:07:55.634135 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6d2a4a-99f1-4280-8eeb-10c9bb5a6ba5 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:07:55.634593 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6d2a4a-99f1-4280-8eeb-10c9bb5a6ba5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:07:55.685539 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-3d6d2a4a-99f1-4280-8eeb-10c9bb5a6ba5 None None] MFA Rules not processed for user `fef422264d0c4a43a0aa1b9eff2251bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:07:55.687495 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-3d6d2a4a-99f1-4280-8eeb-10c9bb5a6ba5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:07:55.716924 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 830/1693] 10.222.0.46 () {60 vars in 1153 bytes} [Mon Oct 10 12:07:55 2022] POST /identity/v3/projects => generated 383 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 12:07:55.721118 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa26d093-7a0e-4979-9e15-baecef2af796 admin admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:07:55.789626 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 864/1694] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:07:55 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 10 12:07:55.795739 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6d2a4a-99f1-4280-8eeb-10c9bb5a6ba5 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:07:55.798458 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-3d6d2a4a-99f1-4280-8eeb-10c9bb5a6ba5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:07:55.875393 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa26d093-7a0e-4979-9e15-baecef2af796 admin admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:07:55.888951 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb083e0b-3fa5-4d9e-95df-96bd8df91e51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:07:55.889606 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb083e0b-3fa5-4d9e-95df-96bd8df91e51 admin admin] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:07:55.889888 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb083e0b-3fa5-4d9e-95df-96bd8df91e51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:07:55.890168 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb083e0b-3fa5-4d9e-95df-96bd8df91e51 admin admin] PATH_INFO: `/v3/users` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:07:55.891013 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb083e0b-3fa5-4d9e-95df-96bd8df91e51 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:07:55.893070 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb083e0b-3fa5-4d9e-95df-96bd8df91e51 admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:07:55.930941 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:07:56.002197 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6d2a4a-99f1-4280-8eeb-10c9bb5a6ba5 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:07:56.087917 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 831/1695] 10.222.0.46 () {60 vars in 1144 bytes} [Mon Oct 10 12:07:55 2022] POST /identity/v3/users => generated 292 bytes in 367 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 10 12:07:56.099289 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb083e0b-3fa5-4d9e-95df-96bd8df91e51 admin admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:07:56.112196 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772cec9f-f05a-4a09-ab37-e49bd9be6eb9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f130bbb69e546ce99b4e14de81d7e54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad3772f7b6a47c28b7ba37a9316a3c1', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:07:56.112829 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb083e0b-3fa5-4d9e-95df-96bd8df91e51 admin admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:07:56.112990 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772cec9f-f05a-4a09-ab37-e49bd9be6eb9 service nova] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:07:56.113396 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772cec9f-f05a-4a09-ab37-e49bd9be6eb9 service nova] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:07:56.113743 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772cec9f-f05a-4a09-ab37-e49bd9be6eb9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:07:56.123382 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c548bc-f397-455e-b1b7-8f0c2c3b5255 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:07:56.124108 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c548bc-f397-455e-b1b7-8f0c2c3b5255 admin admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:07:56.124702 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c548bc-f397-455e-b1b7-8f0c2c3b5255 admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:07:56.125260 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c548bc-f397-455e-b1b7-8f0c2c3b5255 admin admin] PATH_INFO: `/v3/roles` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:07:56.128263 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c548bc-f397-455e-b1b7-8f0c2c3b5255 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:07:56.133369 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c548bc-f397-455e-b1b7-8f0c2c3b5255 admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:07:56.144371 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 832/1696] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 10 12:07:56 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 12:07:56.150340 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c548bc-f397-455e-b1b7-8f0c2c3b5255 admin admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:07:56.171961 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c548bc-f397-455e-b1b7-8f0c2c3b5255 admin admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:07:56.185410 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee93da5a-88c8-4ee2-beeb-c533cc0b7e29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:07:56.185410 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee93da5a-88c8-4ee2-beeb-c533cc0b7e29 admin admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:07:56.185410 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee93da5a-88c8-4ee2-beeb-c533cc0b7e29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:07:56.185410 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee93da5a-88c8-4ee2-beeb-c533cc0b7e29 admin admin] PATH_INFO: `/v3/roles` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:07:56.187184 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee93da5a-88c8-4ee2-beeb-c533cc0b7e29 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:07:56.189070 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee93da5a-88c8-4ee2-beeb-c533cc0b7e29 admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:07:56.205370 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 833/1697] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 10 12:07:56 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 12:07:56.209450 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee93da5a-88c8-4ee2-beeb-c533cc0b7e29 admin admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:07:56.224270 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee93da5a-88c8-4ee2-beeb-c533cc0b7e29 admin admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:07:56.237144 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972133e7-62ae-47df-aa62-3986e1462b92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:07:56.237903 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972133e7-62ae-47df-aa62-3986e1462b92 admin admin] REQUEST_METHOD: `PUT` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:07:56.238188 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972133e7-62ae-47df-aa62-3986e1462b92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:07:56.238465 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972133e7-62ae-47df-aa62-3986e1462b92 admin admin] PATH_INFO: `/v3/projects/94abcb2bea8e40f1ba4d1197f22673aa/users/e26432aea0c14ec5ad8ab72935560b7a/roles/b2cb2a6e818e47338ce071e6fedfc673` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:07:56.243080 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772cec9f-f05a-4a09-ab37-e49bd9be6eb9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:07:56.246101 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772cec9f-f05a-4a09-ab37-e49bd9be6eb9 service nova] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:07:56.261514 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972133e7-62ae-47df-aa62-3986e1462b92 admin admin] RBAC: Authorizing `identity:create_grant(project_id=94abcb2bea8e40f1ba4d1197f22673aa, user_id=e26432aea0c14ec5ad8ab72935560b7a, role_id=b2cb2a6e818e47338ce071e6fedfc673)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:07:56.264571 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972133e7-62ae-47df-aa62-3986e1462b92 admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:07:56.277799 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 865/1698] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 10 12:07:55 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 482 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:07:56.380961 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 834/1699] 10.222.0.46 () {60 vars in 1483 bytes} [Mon Oct 10 12:07:56 2022] PUT /identity/v3/projects/94abcb2bea8e40f1ba4d1197f22673aa/users/e26432aea0c14ec5ad8ab72935560b7a/roles/b2cb2a6e818e47338ce071e6fedfc673 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 10 12:07:56.384099 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbb19bb-3c47-487a-a13a-b6cf69235456 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:07:56.385245 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbb19bb-3c47-487a-a13a-b6cf69235456 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:07:56.386086 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbb19bb-3c47-487a-a13a-b6cf69235456 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:07:56.452561 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-1cbb19bb-3c47-487a-a13a-b6cf69235456 None None] MFA Rules not processed for user `e26432aea0c14ec5ad8ab72935560b7a`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:07:56.454058 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-1cbb19bb-3c47-487a-a13a-b6cf69235456 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:07:56.642002 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 866/1700] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:07:56 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 261 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:07:56.647007 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972133e7-62ae-47df-aa62-3986e1462b92 admin admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:07:56.769431 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:DELETE=6,keystone:SELECT=88,keystone:INSERT=6 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:07:56.808126 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbb19bb-3c47-487a-a13a-b6cf69235456 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:07:56.809653 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-1cbb19bb-3c47-487a-a13a-b6cf69235456 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:07:56.822299 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972133e7-62ae-47df-aa62-3986e1462b92 admin admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:07:56.834788 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbb19bb-3c47-487a-a13a-b6cf69235456 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:07:56.852591 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab49a6e8-dbcb-42d0-b6d6-208453e1c533 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:07:56.853273 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab49a6e8-dbcb-42d0-b6d6-208453e1c533 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:07:56.853816 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab49a6e8-dbcb-42d0-b6d6-208453e1c533 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:07:56.854091 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab49a6e8-dbcb-42d0-b6d6-208453e1c533 admin admin] PATH_INFO: `/v3/projects/b8bbbb12912d456c82b35bac26a7d1a9` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:07:56.857337 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef7513b-4f01-4ba8-841c-08a6c4fd8279 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:07:56.858161 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef7513b-4f01-4ba8-841c-08a6c4fd8279 admin admin] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:07:56.858776 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef7513b-4f01-4ba8-841c-08a6c4fd8279 admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:07:56.859152 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef7513b-4f01-4ba8-841c-08a6c4fd8279 admin admin] PATH_INFO: `/v3/projects` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:07:56.860535 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef7513b-4f01-4ba8-841c-08a6c4fd8279 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:07:56.861307 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab49a6e8-dbcb-42d0-b6d6-208453e1c533 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b8bbbb12912d456c82b35bac26a7d1a9)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:07:56.862938 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef7513b-4f01-4ba8-841c-08a6c4fd8279 admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:07:56.863797 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab49a6e8-dbcb-42d0-b6d6-208453e1c533 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:07:56.874539 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:07:56.985461 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 835/1701] 10.222.0.46 () {60 vars in 1153 bytes} [Mon Oct 10 12:07:56 2022] POST /identity/v3/projects => generated 381 bytes in 339 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 12:07:56.996312 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d94242-101e-4014-a353-eadb5f8f1a89 None None] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:07:56.998011 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d94242-101e-4014-a353-eadb5f8f1a89 None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:07:57.001212 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d94242-101e-4014-a353-eadb5f8f1a89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:07:57.034849 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-ab49a6e8-dbcb-42d0-b6d6-208453e1c533 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b8bbbb12912d456c82b35bac26a7d1a9', 'request_id': 'req-ab49a6e8-dbcb-42d0-b6d6-208453e1c533'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:07:57.051499 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-ab49a6e8-dbcb-42d0-b6d6-208453e1c533 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b8bbbb12912d456c82b35bac26a7d1a9', 'request_id': 'req-ab49a6e8-dbcb-42d0-b6d6-208453e1c533'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:07:57.073430 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-ab49a6e8-dbcb-42d0-b6d6-208453e1c533 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b8bbbb12912d456c82b35bac26a7d1a9', 'request_id': 'req-ab49a6e8-dbcb-42d0-b6d6-208453e1c533'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:07:57.089473 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.auth.core [None req-d9d94242-101e-4014-a353-eadb5f8f1a89 None None] MFA Rules not processed for user `6bc73e7bcf19443a940b6969e1be8e47`. Rule list: `[]` (Enabled: `True`). {{(pid=81676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:07:57.091276 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-d9d94242-101e-4014-a353-eadb5f8f1a89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:07:57.180646 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-ab49a6e8-dbcb-42d0-b6d6-208453e1c533 admin admin] The token cache is being invalidate because project b8bbbb12912d456c82b35bac26a7d1a9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81677) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 10 12:07:57.181150 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-ab49a6e8-dbcb-42d0-b6d6-208453e1c533 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:07:57.182094 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-ab49a6e8-dbcb-42d0-b6d6-208453e1c533 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:07:57.188206 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 867/1702] 10.222.0.46 () {58 vars in 1233 bytes} [Mon Oct 10 12:07:56 2022] DELETE /identity/v3/projects/b8bbbb12912d456c82b35bac26a7d1a9 => generated 0 bytes in 377 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:07:57.189375 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab49a6e8-dbcb-42d0-b6d6-208453e1c533 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:07:57.195379 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-ab49a6e8-dbcb-42d0-b6d6-208453e1c533 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:07:57.249275 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 836/1703] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:07:56 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 263 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:07:57.250278 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d94242-101e-4014-a353-eadb5f8f1a89 None None] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:07:57.251938 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-d9d94242-101e-4014-a353-eadb5f8f1a89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:07:57.360233 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab49a6e8-dbcb-42d0-b6d6-208453e1c533 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:07:57.369427 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c56f6ca-ae61-40f0-ba5e-8e5613ee9644 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:07:57.369427 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c56f6ca-ae61-40f0-ba5e-8e5613ee9644 admin admin] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:07:57.369427 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c56f6ca-ae61-40f0-ba5e-8e5613ee9644 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:07:57.369427 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c56f6ca-ae61-40f0-ba5e-8e5613ee9644 admin admin] PATH_INFO: `/v3/users` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:07:57.369991 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c56f6ca-ae61-40f0-ba5e-8e5613ee9644 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:07:57.372360 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c56f6ca-ae61-40f0-ba5e-8e5613ee9644 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:07:57.426264 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d94242-101e-4014-a353-eadb5f8f1a89 None None] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:07:57.437561 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084e411a-d805-486d-951e-76af9ed4a819 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:07:57.439236 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084e411a-d805-486d-951e-76af9ed4a819 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:07:57.439519 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084e411a-d805-486d-951e-76af9ed4a819 admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:07:57.439872 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084e411a-d805-486d-951e-76af9ed4a819 admin admin] PATH_INFO: `/v3/users/cbd1e94ae9a44de3bf190ef8959961b4` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:07:57.441467 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084e411a-d805-486d-951e-76af9ed4a819 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cbd1e94ae9a44de3bf190ef8959961b4)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:07:57.444316 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084e411a-d805-486d-951e-76af9ed4a819 admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:07:57.500858 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 868/1704] 10.222.0.46 () {60 vars in 1144 bytes} [Mon Oct 10 12:07:57 2022] POST /identity/v3/users => generated 297 bytes in 312 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 12:07:57.502499 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcdfadb-c1a5-4ffc-b39a-ed49a8c0c663 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:07:57.503845 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcdfadb-c1a5-4ffc-b39a-ed49a8c0c663 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:07:57.510905 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcdfadb-c1a5-4ffc-b39a-ed49a8c0c663 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:07:57.579447 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-ffcdfadb-c1a5-4ffc-b39a-ed49a8c0c663 None None] MFA Rules not processed for user `6bc73e7bcf19443a940b6969e1be8e47`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:07:57.584196 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-ffcdfadb-c1a5-4ffc-b39a-ed49a8c0c663 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:07:57.614013 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 869/1705] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:07:57 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:07:57.614262 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffcdfadb-c1a5-4ffc-b39a-ed49a8c0c663 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:07:57.645500 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffcdfadb-c1a5-4ffc-b39a-ed49a8c0c663 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:07:57.659217 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2139d32a-9eee-49be-a81d-6758d7f47393 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:07:57.660136 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2139d32a-9eee-49be-a81d-6758d7f47393 admin admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:07:57.660565 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2139d32a-9eee-49be-a81d-6758d7f47393 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:07:57.661674 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2139d32a-9eee-49be-a81d-6758d7f47393 admin admin] PATH_INFO: `/v3/roles` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:07:57.662432 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2139d32a-9eee-49be-a81d-6758d7f47393 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:07:57.664674 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2139d32a-9eee-49be-a81d-6758d7f47393 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:07:57.704833 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-084e411a-d805-486d-951e-76af9ed4a819 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cbd1e94ae9a44de3bf190ef8959961b4'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:07:57.732247 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 870/1706] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 10 12:07:57 2022] GET /identity/v3/roles => generated 1420 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 12:07:57.733211 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2139d32a-9eee-49be-a81d-6758d7f47393 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:07:57.738347 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-2139d32a-9eee-49be-a81d-6758d7f47393 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:07:57.743344 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-084e411a-d805-486d-951e-76af9ed4a819 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cbd1e94ae9a44de3bf190ef8959961b4'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:07:57.744819 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-084e411a-d805-486d-951e-76af9ed4a819 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cbd1e94ae9a44de3bf190ef8959961b4'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:07:57.851774 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-084e411a-d805-486d-951e-76af9ed4a819 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cbd1e94ae9a44de3bf190ef8959961b4'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:07:57.877987 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-084e411a-d805-486d-951e-76af9ed4a819 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cbd1e94ae9a44de3bf190ef8959961b4'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:07:57.891946 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 837/1707] 10.222.0.46 () {58 vars in 1224 bytes} [Mon Oct 10 12:07:57 2022] DELETE /identity/v3/users/cbd1e94ae9a44de3bf190ef8959961b4 => generated 0 bytes in 642 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:07:57.894318 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d4b290-94b6-4c44-90b6-ff3dd679444b None None] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:07:57.914454 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d4b290-94b6-4c44-90b6-ff3dd679444b None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:07:57.914454 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d4b290-94b6-4c44-90b6-ff3dd679444b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:07:57.938562 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=92,keystone:INSERT=1,keystone:DELETE=7 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:07:57.994731 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:07:58.010515 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.auth.core [None req-30d4b290-94b6-4c44-90b6-ff3dd679444b None None] MFA Rules not processed for user `45c87aa85b7d45fe8acf6f36a3d2a639`. Rule list: `[]` (Enabled: `True`). {{(pid=81676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:07:58.015875 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-30d4b290-94b6-4c44-90b6-ff3dd679444b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:07:58.024487 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2139d32a-9eee-49be-a81d-6758d7f47393 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:07:58.190393 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c0973c-4550-4d42-9231-7fbebe52e131 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f130bbb69e546ce99b4e14de81d7e54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad3772f7b6a47c28b7ba37a9316a3c1', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:07:58.191516 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c0973c-4550-4d42-9231-7fbebe52e131 service nova] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:07:58.192100 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c0973c-4550-4d42-9231-7fbebe52e131 service nova] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:07:58.192484 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c0973c-4550-4d42-9231-7fbebe52e131 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:07:58.196862 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-a8c0973c-4550-4d42-9231-7fbebe52e131 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:07:58.269947 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 838/1708] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:07:57 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 377 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:07:58.271055 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d4b290-94b6-4c44-90b6-ff3dd679444b None None] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:07:58.274977 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-30d4b290-94b6-4c44-90b6-ff3dd679444b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:07:58.371538 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c0973c-4550-4d42-9231-7fbebe52e131 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:07:58.378612 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c0973c-4550-4d42-9231-7fbebe52e131 service nova] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:07:58.421511 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 871/1709] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 10 12:07:57 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 688 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:07:58.426770 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfa7322-62a9-4cbd-86c2-3f47fa18d470 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:07:58.427111 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfa7322-62a9-4cbd-86c2-3f47fa18d470 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:07:58.427392 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfa7322-62a9-4cbd-86c2-3f47fa18d470 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:07:58.466612 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d4b290-94b6-4c44-90b6-ff3dd679444b None None] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:07:58.481516 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efef276d-04f5-4202-91c5-57785c14c47d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:07:58.482436 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efef276d-04f5-4202-91c5-57785c14c47d admin admin] REQUEST_METHOD: `PUT` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:07:58.482660 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efef276d-04f5-4202-91c5-57785c14c47d admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:07:58.483057 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efef276d-04f5-4202-91c5-57785c14c47d admin admin] PATH_INFO: `/v3/projects/9d8f087abfc148fe985d45c5a92f837b/users/d3b878cff9164371a0944d233021265e/roles/0d822a0674b6495fac623877a3b38025` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:07:58.514903 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-6bfa7322-62a9-4cbd-86c2-3f47fa18d470 None None] MFA Rules not processed for user `b334b2401e14419284cce2ad61442d4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:07:58.516459 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-6bfa7322-62a9-4cbd-86c2-3f47fa18d470 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:07:58.517147 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efef276d-04f5-4202-91c5-57785c14c47d admin admin] RBAC: Authorizing `identity:create_grant(project_id=9d8f087abfc148fe985d45c5a92f837b, user_id=d3b878cff9164371a0944d233021265e, role_id=0d822a0674b6495fac623877a3b38025)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:07:58.522230 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efef276d-04f5-4202-91c5-57785c14c47d admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:07:58.611072 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 839/1710] 10.222.0.46 () {60 vars in 1483 bytes} [Mon Oct 10 12:07:58 2022] PUT /identity/v3/projects/9d8f087abfc148fe985d45c5a92f837b/users/d3b878cff9164371a0944d233021265e/roles/0d822a0674b6495fac623877a3b38025 => generated 0 bytes in 341 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:07:58.616029 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d55f1db-ff60-4331-9178-5e8642450f14 None None] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:07:58.616708 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d55f1db-ff60-4331-9178-5e8642450f14 None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:07:58.617381 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d55f1db-ff60-4331-9178-5e8642450f14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:07:58.655857 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 872/1711] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:07:58 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:07:58.657335 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfa7322-62a9-4cbd-86c2-3f47fa18d470 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:07:58.708663 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfa7322-62a9-4cbd-86c2-3f47fa18d470 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:07:58.708892 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.auth.core [None req-9d55f1db-ff60-4331-9178-5e8642450f14 None None] MFA Rules not processed for user `45c87aa85b7d45fe8acf6f36a3d2a639`. Rule list: `[]` (Enabled: `True`). {{(pid=81676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:07:58.714003 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-9d55f1db-ff60-4331-9178-5e8642450f14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:07:58.825415 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:07:58.882455 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:07:58.938374 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe80aaf-7fb9-4728-853b-de7673171db5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:07:58.939242 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe80aaf-7fb9-4728-853b-de7673171db5 admin admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:07:58.939527 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe80aaf-7fb9-4728-853b-de7673171db5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:07:58.939809 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe80aaf-7fb9-4728-853b-de7673171db5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:07:58.940460 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe80aaf-7fb9-4728-853b-de7673171db5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:07:58.946408 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe80aaf-7fb9-4728-853b-de7673171db5 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:07:58.969170 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 873/1712] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 10 12:07:58 2022] GET /identity/v3/roles => generated 1420 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 12:07:58.970103 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe80aaf-7fb9-4728-853b-de7673171db5 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:07:58.977160 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-6fe80aaf-7fb9-4728-853b-de7673171db5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:07:59.010053 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 840/1713] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:07:58 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 398 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:07:59.012345 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e8c489-33b8-4a9d-b916-b5c4fcdae214 None None] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:07:59.013148 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e8c489-33b8-4a9d-b916-b5c4fcdae214 None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:07:59.013148 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e8c489-33b8-4a9d-b916-b5c4fcdae214 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:07:59.144042 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.auth.core [None req-18e8c489-33b8-4a9d-b916-b5c4fcdae214 None None] MFA Rules not processed for user `a7a87488f78444c3ad820527e5fbc099`. Rule list: `[]` (Enabled: `True`). {{(pid=81676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:07:59.145391 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-18e8c489-33b8-4a9d-b916-b5c4fcdae214 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:07:59.279770 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe80aaf-7fb9-4728-853b-de7673171db5 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:07:59.301753 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9ed69f-5e5a-408e-a29e-133cc3bc2329 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f130bbb69e546ce99b4e14de81d7e54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad3772f7b6a47c28b7ba37a9316a3c1', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:07:59.302396 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9ed69f-5e5a-408e-a29e-133cc3bc2329 service nova] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:07:59.302702 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9ed69f-5e5a-408e-a29e-133cc3bc2329 service nova] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:07:59.303009 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9ed69f-5e5a-408e-a29e-133cc3bc2329 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:07:59.377393 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 841/1714] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:07:59 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 367 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:07:59.389816 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e8c489-33b8-4a9d-b916-b5c4fcdae214 None None] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:07:59.423828 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e8c489-33b8-4a9d-b916-b5c4fcdae214 None None] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:07:59.434006 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5f01b8-a365-41ca-a190-1ecf4ac0e629 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:07:59.437596 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5f01b8-a365-41ca-a190-1ecf4ac0e629 admin admin] REQUEST_METHOD: `PUT` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:07:59.438190 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5f01b8-a365-41ca-a190-1ecf4ac0e629 admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:07:59.438471 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5f01b8-a365-41ca-a190-1ecf4ac0e629 admin admin] PATH_INFO: `/v3/projects/9d8f087abfc148fe985d45c5a92f837b/users/d3b878cff9164371a0944d233021265e/roles/0d822a0674b6495fac623877a3b38025` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:07:59.442107 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5f01b8-a365-41ca-a190-1ecf4ac0e629 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9d8f087abfc148fe985d45c5a92f837b, user_id=d3b878cff9164371a0944d233021265e, role_id=0d822a0674b6495fac623877a3b38025)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:07:59.453759 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5f01b8-a365-41ca-a190-1ecf4ac0e629 admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:07:59.494238 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 842/1715] 10.222.0.46 () {60 vars in 1483 bytes} [Mon Oct 10 12:07:59 2022] PUT /identity/v3/projects/9d8f087abfc148fe985d45c5a92f837b/users/d3b878cff9164371a0944d233021265e/roles/0d822a0674b6495fac623877a3b38025 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:07:59.507682 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cd88bc-3250-4750-8bf8-da7dd735e0d9 None None] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:07:59.507961 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cd88bc-3250-4750-8bf8-da7dd735e0d9 None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:07:59.508240 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cd88bc-3250-4750-8bf8-da7dd735e0d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:07:59.524903 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9ed69f-5e5a-408e-a29e-133cc3bc2329 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:07:59.527594 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9ed69f-5e5a-408e-a29e-133cc3bc2329 service nova] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:07:59.548729 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 874/1716] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 10 12:07:58 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 579 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:07:59.595624 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.auth.core [None req-15cd88bc-3250-4750-8bf8-da7dd735e0d9 None None] MFA Rules not processed for user `d3b878cff9164371a0944d233021265e`. Rule list: `[]` (Enabled: `True`). {{(pid=81676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:07:59.598989 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-15cd88bc-3250-4750-8bf8-da7dd735e0d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:07:59.613516 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9ed69f-5e5a-408e-a29e-133cc3bc2329 service nova] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:07:59.615197 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-df9ed69f-5e5a-408e-a29e-133cc3bc2329 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:07:59.649633 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9ed69f-5e5a-408e-a29e-133cc3bc2329 service nova] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:07:59.763978 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 843/1717] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:07:59 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 257 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:07:59.769079 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eea833b-1c15-46b1-bc96-ed9ad88a9203 None None] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:07:59.769379 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eea833b-1c15-46b1-bc96-ed9ad88a9203 None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:07:59.769659 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eea833b-1c15-46b1-bc96-ed9ad88a9203 None None] PATH_INFO: `/` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:07:59.779328 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 844/1718] 10.222.0.46 () {54 vars in 912 bytes} [Mon Oct 10 12:07:59 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 12:07:59.779328 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eea833b-1c15-46b1-bc96-ed9ad88a9203 None None] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:07:59.783192 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-7eea833b-1c15-46b1-bc96-ed9ad88a9203 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:07:59.789333 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09cfca3b-f8fc-4e9d-ae82-e4c969646b95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:07:59.790448 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cfca3b-f8fc-4e9d-ae82-e4c969646b95 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:07:59.790915 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cfca3b-f8fc-4e9d-ae82-e4c969646b95 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:07:59.791663 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cfca3b-f8fc-4e9d-ae82-e4c969646b95 admin admin] PATH_INFO: `/v3/projects/67dbb5f9c61a4cf0883d7f5376f3a0af` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:07:59.793968 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09cfca3b-f8fc-4e9d-ae82-e4c969646b95 admin admin] RBAC: Authorizing `identity:delete_project(project_id=67dbb5f9c61a4cf0883d7f5376f3a0af)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:07:59.796812 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09cfca3b-f8fc-4e9d-ae82-e4c969646b95 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:07:59.811550 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:07:59.903637 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eea833b-1c15-46b1-bc96-ed9ad88a9203 None None] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:07:59.907542 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3aa391-60da-4f03-b874-ad5fd11df476 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3ae093470484b93b23dcfae1657df9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad3772f7b6a47c28b7ba37a9316a3c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:07:59.910057 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3aa391-60da-4f03-b874-ad5fd11df476 service cinder] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:07:59.910467 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3aa391-60da-4f03-b874-ad5fd11df476 service cinder] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:07:59.910911 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3aa391-60da-4f03-b874-ad5fd11df476 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:07:59.918181 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:07:59.926647 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3aa391-60da-4f03-b874-ad5fd11df476 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:07:59.928305 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3aa391-60da-4f03-b874-ad5fd11df476 service cinder] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:07:59.940412 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 845/1719] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 10 12:07:59 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:07:59.953212 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3aa391-60da-4f03-b874-ad5fd11df476 service cinder] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:07:59.997039 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-09cfca3b-f8fc-4e9d-ae82-e4c969646b95 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '67dbb5f9c61a4cf0883d7f5376f3a0af', 'request_id': 'req-09cfca3b-f8fc-4e9d-ae82-e4c969646b95'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:08:00.008274 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-09cfca3b-f8fc-4e9d-ae82-e4c969646b95 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '67dbb5f9c61a4cf0883d7f5376f3a0af', 'request_id': 'req-09cfca3b-f8fc-4e9d-ae82-e4c969646b95'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:08:00.031611 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-09cfca3b-f8fc-4e9d-ae82-e4c969646b95 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '67dbb5f9c61a4cf0883d7f5376f3a0af', 'request_id': 'req-09cfca3b-f8fc-4e9d-ae82-e4c969646b95'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:08:00.095226 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3aa391-60da-4f03-b874-ad5fd11df476 service cinder] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:08:00.127699 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-09cfca3b-f8fc-4e9d-ae82-e4c969646b95 admin admin] The token cache is being invalidate because project 67dbb5f9c61a4cf0883d7f5376f3a0af was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81677) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 10 12:08:00.127699 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-09cfca3b-f8fc-4e9d-ae82-e4c969646b95 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:08:00.127699 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-09cfca3b-f8fc-4e9d-ae82-e4c969646b95 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:08:00.133151 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 875/1720] 10.222.0.46 () {58 vars in 1233 bytes} [Mon Oct 10 12:07:59 2022] DELETE /identity/v3/projects/67dbb5f9c61a4cf0883d7f5376f3a0af => generated 0 bytes in 513 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 10 12:08:00.134920 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b8567f-e831-426f-9d5c-09897f12d518 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:08:00.135656 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b8567f-e831-426f-9d5c-09897f12d518 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:08:00.136199 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b8567f-e831-426f-9d5c-09897f12d518 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:08:00.235056 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-f3b8567f-e831-426f-9d5c-09897f12d518 None None] MFA Rules not processed for user `58b1c7c85302421cba823ace4201861f`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:08:00.237169 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-f3b8567f-e831-426f-9d5c-09897f12d518 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:08:00.302092 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7184a2-2c72-4079-9d35-33c1b02d6f1a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f130bbb69e546ce99b4e14de81d7e54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad3772f7b6a47c28b7ba37a9316a3c1', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:08:00.303546 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7184a2-2c72-4079-9d35-33c1b02d6f1a service nova] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:08:00.303942 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7184a2-2c72-4079-9d35-33c1b02d6f1a service nova] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:08:00.304329 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7184a2-2c72-4079-9d35-33c1b02d6f1a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:08:00.306823 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-0c7184a2-2c72-4079-9d35-33c1b02d6f1a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:08:00.356077 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 876/1721] 10.222.0.46 () {58 vars in 1009 bytes} [Mon Oct 10 12:08:00 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:08:00.358584 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c3e638-99be-4fa6-8414-e2f406e8299f None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:08:00.358905 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c3e638-99be-4fa6-8414-e2f406e8299f None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:08:00.359179 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c3e638-99be-4fa6-8414-e2f406e8299f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:08:00.392817 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-b2c3e638-99be-4fa6-8414-e2f406e8299f None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:08:00.396701 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-b2c3e638-99be-4fa6-8414-e2f406e8299f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:08:00.413881 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7184a2-2c72-4079-9d35-33c1b02d6f1a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:08:00.417151 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7184a2-2c72-4079-9d35-33c1b02d6f1a service nova] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:08:00.441885 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 846/1722] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 10 12:07:59 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 491 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:08:00.443246 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7184a2-2c72-4079-9d35-33c1b02d6f1a service nova] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:08:00.448712 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-0c7184a2-2c72-4079-9d35-33c1b02d6f1a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:08:00.571647 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 877/1723] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:08:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:08:00.575084 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c3e638-99be-4fa6-8414-e2f406e8299f None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:08:00.578225 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-b2c3e638-99be-4fa6-8414-e2f406e8299f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:08:00.606930 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7184a2-2c72-4079-9d35-33c1b02d6f1a service nova] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:08:00.607447 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c3e638-99be-4fa6-8414-e2f406e8299f None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:08:00.617711 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c60d7e4-8ee6-4908-9cb5-03f6653a52d3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3ae093470484b93b23dcfae1657df9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad3772f7b6a47c28b7ba37a9316a3c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:08:00.619217 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c60d7e4-8ee6-4908-9cb5-03f6653a52d3 service cinder] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:08:00.620505 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c60d7e4-8ee6-4908-9cb5-03f6653a52d3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:08:00.620892 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c60d7e4-8ee6-4908-9cb5-03f6653a52d3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:08:00.626098 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc9e9e7-9cab-4695-9756-742a55a6352f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58b1c7c85302421cba823ace4201861f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad3772f7b6a47c28b7ba37a9316a3c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:08:00.626340 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-1c60d7e4-8ee6-4908-9cb5-03f6653a52d3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:08:00.627120 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc9e9e7-9cab-4695-9756-742a55a6352f service neutron] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:08:00.627551 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc9e9e7-9cab-4695-9756-742a55a6352f service neutron] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:08:00.628013 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc9e9e7-9cab-4695-9756-742a55a6352f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:08:00.640235 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc9e9e7-9cab-4695-9756-742a55a6352f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:08:00.642179 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc9e9e7-9cab-4695-9756-742a55a6352f service neutron] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:08:00.656957 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 878/1724] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 10 12:08:00 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:08:00.658607 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4497ab7-550e-46f0-8f21-887446513868 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:08:00.659073 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4497ab7-550e-46f0-8f21-887446513868 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:08:00.659556 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4497ab7-550e-46f0-8f21-887446513868 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:08:00.666184 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:08:00.708588 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-b4497ab7-550e-46f0-8f21-887446513868 None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:08:00.709550 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-b4497ab7-550e-46f0-8f21-887446513868 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:08:00.742472 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 879/1725] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:08:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:08:00.746756 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4497ab7-550e-46f0-8f21-887446513868 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:08:00.748351 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c60d7e4-8ee6-4908-9cb5-03f6653a52d3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:08:00.750450 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c60d7e4-8ee6-4908-9cb5-03f6653a52d3 service cinder] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:08:00.764939 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 847/1726] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 10 12:08:00 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:08:00.766723 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4497ab7-550e-46f0-8f21-887446513868 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:08:00.780074 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65eff156-c58b-4dbe-b9d4-73ed32310993 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:08:00.780715 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65eff156-c58b-4dbe-b9d4-73ed32310993 admin admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:08:00.780999 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65eff156-c58b-4dbe-b9d4-73ed32310993 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:08:00.781274 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65eff156-c58b-4dbe-b9d4-73ed32310993 admin admin] PATH_INFO: `/v3/domains` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:08:00.781947 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65eff156-c58b-4dbe-b9d4-73ed32310993 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 12:08:00.782251 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65eff156-c58b-4dbe-b9d4-73ed32310993 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:08:00.783976 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65eff156-c58b-4dbe-b9d4-73ed32310993 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:08:00.788879 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:08:00.815266 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 880/1727] 10.222.0.46 () {58 vars in 1153 bytes} [Mon Oct 10 12:08:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 12:08:00.821435 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c60d7e4-8ee6-4908-9cb5-03f6653a52d3 service cinder] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:08:00.853184 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c60d7e4-8ee6-4908-9cb5-03f6653a52d3 service cinder] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:08:00.875628 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f8eee2-4c6e-4837-9fe3-027360f20825 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:08:00.875960 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f8eee2-4c6e-4837-9fe3-027360f20825 admin admin] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:08:00.876239 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f8eee2-4c6e-4837-9fe3-027360f20825 admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:08:00.876514 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f8eee2-4c6e-4837-9fe3-027360f20825 admin admin] PATH_INFO: `/v3/projects` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:08:00.877157 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f8eee2-4c6e-4837-9fe3-027360f20825 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:08:00.888427 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f8eee2-4c6e-4837-9fe3-027360f20825 admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:08:01.031832 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 848/1728] 10.222.0.46 () {60 vars in 1153 bytes} [Mon Oct 10 12:08:00 2022] POST /identity/v3/projects => generated 387 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 12:08:01.035884 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65eff156-c58b-4dbe-b9d4-73ed32310993 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:08:01.050203 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65eff156-c58b-4dbe-b9d4-73ed32310993 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:08:01.125625 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d89cd8-deb4-4bd7-8394-706a07fbd7cc None None] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:08:01.126143 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d89cd8-deb4-4bd7-8394-706a07fbd7cc None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:08:01.126217 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d89cd8-deb4-4bd7-8394-706a07fbd7cc None None] PATH_INFO: `/` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:08:01.127813 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 849/1729] 10.222.0.46 () {54 vars in 912 bytes} [Mon Oct 10 12:08:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 12:08:01.139189 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af90c547-f504-4b27-9dcd-f3c019320757 None None] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:08:01.139464 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af90c547-f504-4b27-9dcd-f3c019320757 None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:08:01.139729 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af90c547-f504-4b27-9dcd-f3c019320757 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:08:01.179756 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.auth.core [None req-af90c547-f504-4b27-9dcd-f3c019320757 None None] MFA Rules not processed for user `58b1c7c85302421cba823ace4201861f`. Rule list: `[]` (Enabled: `True`). {{(pid=81676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:08:01.184378 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-af90c547-f504-4b27-9dcd-f3c019320757 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:08:01.211425 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ed04e4-7a62-4231-be44-8f89468bca7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:08:01.212066 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ed04e4-7a62-4231-be44-8f89468bca7f admin admin] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:08:01.212342 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ed04e4-7a62-4231-be44-8f89468bca7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:08:01.214145 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ed04e4-7a62-4231-be44-8f89468bca7f admin admin] PATH_INFO: `/v3/users` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:08:01.214857 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ed04e4-7a62-4231-be44-8f89468bca7f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:08:01.220721 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ed04e4-7a62-4231-be44-8f89468bca7f admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:08:01.303111 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 850/1730] 10.222.0.46 () {58 vars in 1009 bytes} [Mon Oct 10 12:08:01 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (3 switches on core 0) Oct 10 12:08:01.332017 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 881/1731] 10.222.0.46 () {60 vars in 1144 bytes} [Mon Oct 10 12:08:01 2022] POST /identity/v3/users => generated 294 bytes in 296 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 12:08:01.336514 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af90c547-f504-4b27-9dcd-f3c019320757 None None] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:08:01.360082 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af90c547-f504-4b27-9dcd-f3c019320757 None None] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:08:01.370175 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ed04e4-7a62-4231-be44-8f89468bca7f admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:08:01.372383 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66dce61-c8bd-4add-b467-588118c8ba91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:08:01.373186 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66dce61-c8bd-4add-b467-588118c8ba91 admin admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:08:01.373370 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66dce61-c8bd-4add-b467-588118c8ba91 admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:08:01.373649 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66dce61-c8bd-4add-b467-588118c8ba91 admin admin] PATH_INFO: `/v3/roles` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:08:01.374350 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66dce61-c8bd-4add-b467-588118c8ba91 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:08:01.375990 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66dce61-c8bd-4add-b467-588118c8ba91 admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:08:01.387427 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ed04e4-7a62-4231-be44-8f89468bca7f admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:08:01.393903 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 851/1732] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 10 12:08:01 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 12:08:01.396308 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6b7c76-c907-461f-9d95-753b08f356ab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58b1c7c85302421cba823ace4201861f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad3772f7b6a47c28b7ba37a9316a3c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:08:01.398362 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66dce61-c8bd-4add-b467-588118c8ba91 admin admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:08:01.398554 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6b7c76-c907-461f-9d95-753b08f356ab service neutron] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:08:01.400767 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6b7c76-c907-461f-9d95-753b08f356ab service neutron] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:08:01.402709 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6b7c76-c907-461f-9d95-753b08f356ab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:08:01.417830 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66dce61-c8bd-4add-b467-588118c8ba91 admin admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:08:01.420314 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6b7c76-c907-461f-9d95-753b08f356ab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:08:01.422289 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6b7c76-c907-461f-9d95-753b08f356ab service neutron] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:08:01.439336 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be3b27f-61b0-4920-a266-68ce1f5b2bb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:08:01.439902 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be3b27f-61b0-4920-a266-68ce1f5b2bb9 admin admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:08:01.440179 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be3b27f-61b0-4920-a266-68ce1f5b2bb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:08:01.440451 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be3b27f-61b0-4920-a266-68ce1f5b2bb9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:08:01.441103 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be3b27f-61b0-4920-a266-68ce1f5b2bb9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:08:01.442735 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be3b27f-61b0-4920-a266-68ce1f5b2bb9 admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:08:01.455970 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 882/1733] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 10 12:08:01 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 10 12:08:01.471663 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6b7c76-c907-461f-9d95-753b08f356ab service neutron] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:08:01.472457 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 852/1734] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 10 12:08:01 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 10 12:08:01.558188 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6b7c76-c907-461f-9d95-753b08f356ab service neutron] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:08:01.587950 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a029184c-0663-4c91-9120-19151a534887 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:08:01.589303 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a029184c-0663-4c91-9120-19151a534887 admin admin] REQUEST_METHOD: `PUT` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:08:01.589728 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a029184c-0663-4c91-9120-19151a534887 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:08:01.592733 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a029184c-0663-4c91-9120-19151a534887 admin admin] PATH_INFO: `/v3/projects/2bb99b5b040a48adbf28596e626f2dff/users/d9743a442a6444a9aeb1f3b30f4ec10f/roles/b2cb2a6e818e47338ce071e6fedfc673` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:08:01.621738 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a029184c-0663-4c91-9120-19151a534887 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2bb99b5b040a48adbf28596e626f2dff, user_id=d9743a442a6444a9aeb1f3b30f4ec10f, role_id=b2cb2a6e818e47338ce071e6fedfc673)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:08:01.624404 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a029184c-0663-4c91-9120-19151a534887 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:08:01.763004 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 883/1735] 10.222.0.46 () {60 vars in 1483 bytes} [Mon Oct 10 12:08:01 2022] PUT /identity/v3/projects/2bb99b5b040a48adbf28596e626f2dff/users/d9743a442a6444a9aeb1f3b30f4ec10f/roles/b2cb2a6e818e47338ce071e6fedfc673 => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:08:01.781099 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bee857-5f67-4ef4-8cb8-e8c48a324e2c None None] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:08:01.781995 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bee857-5f67-4ef4-8cb8-e8c48a324e2c None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:08:01.782495 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bee857-5f67-4ef4-8cb8-e8c48a324e2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:08:01.877252 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.auth.core [None req-d3bee857-5f67-4ef4-8cb8-e8c48a324e2c None None] MFA Rules not processed for user `d9743a442a6444a9aeb1f3b30f4ec10f`. Rule list: `[]` (Enabled: `True`). {{(pid=81676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:08:01.883812 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-d3bee857-5f67-4ef4-8cb8-e8c48a324e2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:08:01.976530 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:08:02.284975 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 853/1736] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:08:01 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 324 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:08:02.287005 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a029184c-0663-4c91-9120-19151a534887 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:08:02.445983 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a029184c-0663-4c91-9120-19151a534887 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:08:02.472588 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f970cc5-379d-4ffa-b918-be3ff42ca571 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:08:02.473417 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f970cc5-379d-4ffa-b918-be3ff42ca571 admin admin] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:08:02.473877 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f970cc5-379d-4ffa-b918-be3ff42ca571 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:08:02.474290 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f970cc5-379d-4ffa-b918-be3ff42ca571 admin admin] PATH_INFO: `/v3/projects` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:08:02.475174 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f970cc5-379d-4ffa-b918-be3ff42ca571 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:08:02.478301 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f970cc5-379d-4ffa-b918-be3ff42ca571 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:08:02.516689 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bee857-5f67-4ef4-8cb8-e8c48a324e2c None None] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:08:02.518781 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-d3bee857-5f67-4ef4-8cb8-e8c48a324e2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:08:02.625947 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 884/1737] 10.222.0.46 () {60 vars in 1153 bytes} [Mon Oct 10 12:08:02 2022] POST /identity/v3/projects => generated 383 bytes in 365 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 12:08:02.627228 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f970cc5-379d-4ffa-b918-be3ff42ca571 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:08:02.713961 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bee857-5f67-4ef4-8cb8-e8c48a324e2c None None] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:08:02.726889 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:08:02.805527 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f970cc5-379d-4ffa-b918-be3ff42ca571 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:08:02.819372 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622174e0-da70-4f9e-a4b8-eedc3ade4fea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:08:02.827573 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622174e0-da70-4f9e-a4b8-eedc3ade4fea admin admin] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:08:02.827573 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622174e0-da70-4f9e-a4b8-eedc3ade4fea admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:08:02.827573 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622174e0-da70-4f9e-a4b8-eedc3ade4fea admin admin] PATH_INFO: `/v3/users` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:08:02.827573 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622174e0-da70-4f9e-a4b8-eedc3ade4fea admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:08:02.827573 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622174e0-da70-4f9e-a4b8-eedc3ade4fea admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:08:02.883747 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a33b961-03b4-482d-bd77-0fef88ceec12 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f130bbb69e546ce99b4e14de81d7e54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad3772f7b6a47c28b7ba37a9316a3c1', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:08:02.884450 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a33b961-03b4-482d-bd77-0fef88ceec12 service nova] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:08:02.884735 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a33b961-03b4-482d-bd77-0fef88ceec12 service nova] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:08:02.885889 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a33b961-03b4-482d-bd77-0fef88ceec12 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:08:02.889571 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-3a33b961-03b4-482d-bd77-0fef88ceec12 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:08:02.960110 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 885/1738] 10.222.0.46 () {60 vars in 1144 bytes} [Mon Oct 10 12:08:02 2022] POST /identity/v3/users => generated 298 bytes in 333 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 12:08:02.961598 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622174e0-da70-4f9e-a4b8-eedc3ade4fea admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:08:02.963930 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-622174e0-da70-4f9e-a4b8-eedc3ade4fea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:08:03.015546 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a33b961-03b4-482d-bd77-0fef88ceec12 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:08:03.017321 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a33b961-03b4-482d-bd77-0fef88ceec12 service nova] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:08:03.047839 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 854/1739] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 10 12:08:02 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 532 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 10 12:08:03.069820 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526a5cef-4261-49e9-b922-9b5d603a0098 None None] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:08:03.070114 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526a5cef-4261-49e9-b922-9b5d603a0098 None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:08:03.070395 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526a5cef-4261-49e9-b922-9b5d603a0098 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:08:03.125787 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.auth.core [None req-526a5cef-4261-49e9-b922-9b5d603a0098 None None] MFA Rules not processed for user `fef422264d0c4a43a0aa1b9eff2251bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:08:03.129374 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-526a5cef-4261-49e9-b922-9b5d603a0098 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:08:03.161470 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622174e0-da70-4f9e-a4b8-eedc3ade4fea admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:08:03.168482 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4832a1-4d6a-4929-8940-09d5f1327ba7 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e8f9bb1776d47169ef7febb57c954cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad3772f7b6a47c28b7ba37a9316a3c1', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:08:03.169685 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4832a1-4d6a-4929-8940-09d5f1327ba7 service placement] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:08:03.170298 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4832a1-4d6a-4929-8940-09d5f1327ba7 service placement] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:08:03.170946 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4832a1-4d6a-4929-8940-09d5f1327ba7 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:08:03.173857 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-7b4832a1-4d6a-4929-8940-09d5f1327ba7 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:08:03.210920 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4832a1-4d6a-4929-8940-09d5f1327ba7 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:08:03.210920 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4832a1-4d6a-4929-8940-09d5f1327ba7 service placement] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:08:03.223816 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 886/1740] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 10 12:08:02 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 262 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:08:03.224957 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4832a1-4d6a-4929-8940-09d5f1327ba7 service placement] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:08:03.237287 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4832a1-4d6a-4929-8940-09d5f1327ba7 service placement] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:08:03.247811 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700dd0ed-d9f1-4d17-a099-21a31aa80817 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:08:03.248757 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700dd0ed-d9f1-4d17-a099-21a31aa80817 admin admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:08:03.249143 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700dd0ed-d9f1-4d17-a099-21a31aa80817 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:08:03.249521 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700dd0ed-d9f1-4d17-a099-21a31aa80817 admin admin] PATH_INFO: `/v3/roles` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:08:03.250349 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700dd0ed-d9f1-4d17-a099-21a31aa80817 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:08:03.252295 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700dd0ed-d9f1-4d17-a099-21a31aa80817 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:08:03.271473 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 887/1741] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 10 12:08:03 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 12:08:03.273481 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700dd0ed-d9f1-4d17-a099-21a31aa80817 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:08:03.293241 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700dd0ed-d9f1-4d17-a099-21a31aa80817 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:08:03.301009 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e296af-dbf4-4a2f-af9a-1b18c4b881b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:08:03.302375 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e296af-dbf4-4a2f-af9a-1b18c4b881b5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:08:03.311866 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e296af-dbf4-4a2f-af9a-1b18c4b881b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:08:03.314462 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e296af-dbf4-4a2f-af9a-1b18c4b881b5 admin admin] PATH_INFO: `/v3/projects/da7e6d4dceff4b67a5187f5619cac759/users/1f13f0a9eba346a4bef8634e4d82feab/roles/0d822a0674b6495fac623877a3b38025` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:08:03.318192 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:08:03.383129 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 855/1742] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:08:03 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 311 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:08:03.387526 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e296af-dbf4-4a2f-af9a-1b18c4b881b5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=da7e6d4dceff4b67a5187f5619cac759, user_id=1f13f0a9eba346a4bef8634e4d82feab, role_id=0d822a0674b6495fac623877a3b38025)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:08:03.391492 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e296af-dbf4-4a2f-af9a-1b18c4b881b5 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:08:03.505855 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 888/1743] 10.222.0.46 () {60 vars in 1483 bytes} [Mon Oct 10 12:08:03 2022] PUT /identity/v3/projects/da7e6d4dceff4b67a5187f5619cac759/users/1f13f0a9eba346a4bef8634e4d82feab/roles/0d822a0674b6495fac623877a3b38025 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:08:03.509968 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526a5cef-4261-49e9-b922-9b5d603a0098 None None] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:08:03.549421 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526a5cef-4261-49e9-b922-9b5d603a0098 None None] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:08:03.738859 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd2352f-859a-4037-8874-0c5226ef6e06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:08:03.739496 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd2352f-859a-4037-8874-0c5226ef6e06 admin admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:08:03.739781 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd2352f-859a-4037-8874-0c5226ef6e06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:08:03.740061 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd2352f-859a-4037-8874-0c5226ef6e06 admin admin] PATH_INFO: `/v3/roles` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:08:03.740717 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd2352f-859a-4037-8874-0c5226ef6e06 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:08:03.742364 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd2352f-859a-4037-8874-0c5226ef6e06 admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:08:03.758819 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c870edf4-626f-4089-b9b2-cf039bbee1da None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:08:03.759360 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c870edf4-626f-4089-b9b2-cf039bbee1da None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:08:03.759602 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c870edf4-626f-4089-b9b2-cf039bbee1da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:08:03.809744 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 856/1744] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 10 12:08:03 2022] GET /identity/v3/roles => generated 1420 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 12:08:03.810819 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd2352f-859a-4037-8874-0c5226ef6e06 admin admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:08:03.829235 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-c870edf4-626f-4089-b9b2-cf039bbee1da None None] MFA Rules not processed for user `45c87aa85b7d45fe8acf6f36a3d2a639`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:08:03.830639 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-c870edf4-626f-4089-b9b2-cf039bbee1da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:08:03.855375 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd2352f-859a-4037-8874-0c5226ef6e06 admin admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:08:03.878684 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7328fe-02c6-4d89-970c-c45b46fc1ea4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:08:03.881486 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7328fe-02c6-4d89-970c-c45b46fc1ea4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:08:03.882772 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7328fe-02c6-4d89-970c-c45b46fc1ea4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:08:03.883934 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7328fe-02c6-4d89-970c-c45b46fc1ea4 admin admin] PATH_INFO: `/v3/projects/da7e6d4dceff4b67a5187f5619cac759/users/1f13f0a9eba346a4bef8634e4d82feab/roles/0d822a0674b6495fac623877a3b38025` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:08:03.891710 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7328fe-02c6-4d89-970c-c45b46fc1ea4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=da7e6d4dceff4b67a5187f5619cac759, user_id=1f13f0a9eba346a4bef8634e4d82feab, role_id=0d822a0674b6495fac623877a3b38025)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:08:03.895920 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7328fe-02c6-4d89-970c-c45b46fc1ea4 admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:08:03.946043 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 857/1745] 10.222.0.46 () {60 vars in 1483 bytes} [Mon Oct 10 12:08:03 2022] PUT /identity/v3/projects/da7e6d4dceff4b67a5187f5619cac759/users/1f13f0a9eba346a4bef8634e4d82feab/roles/0d822a0674b6495fac623877a3b38025 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:08:03.965033 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f258a47d-0f58-4045-bc54-039e9bc5d296 None None] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:08:03.965033 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f258a47d-0f58-4045-bc54-039e9bc5d296 None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:08:03.965033 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f258a47d-0f58-4045-bc54-039e9bc5d296 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:08:03.993764 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 889/1746] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:08:03 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 10 12:08:04.070413 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.auth.core [None req-f258a47d-0f58-4045-bc54-039e9bc5d296 None None] MFA Rules not processed for user `1f13f0a9eba346a4bef8634e4d82feab`. Rule list: `[]` (Enabled: `True`). {{(pid=81676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:08:04.070413 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-f258a47d-0f58-4045-bc54-039e9bc5d296 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:08:04.301093 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 858/1747] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:08:03 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 336 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:08:04.358953 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c870edf4-626f-4089-b9b2-cf039bbee1da None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:08:04.358953 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-c870edf4-626f-4089-b9b2-cf039bbee1da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:08:04.412421 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:08:04.512172 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c870edf4-626f-4089-b9b2-cf039bbee1da None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:08:04.668317 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946c6ccb-ecc9-4df5-b06a-36658fdf2c7f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3ae093470484b93b23dcfae1657df9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad3772f7b6a47c28b7ba37a9316a3c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:08:04.669035 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946c6ccb-ecc9-4df5-b06a-36658fdf2c7f service cinder] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:08:04.669321 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946c6ccb-ecc9-4df5-b06a-36658fdf2c7f service cinder] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:08:04.669600 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946c6ccb-ecc9-4df5-b06a-36658fdf2c7f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:08:04.697409 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946c6ccb-ecc9-4df5-b06a-36658fdf2c7f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:08:04.703487 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946c6ccb-ecc9-4df5-b06a-36658fdf2c7f service cinder] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:08:04.743334 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 890/1748] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 10 12:08:04 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:08:05.129714 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f258a47d-0f58-4045-bc54-039e9bc5d296 None None] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:08:05.195924 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:08:05.334682 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f258a47d-0f58-4045-bc54-039e9bc5d296 None None] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:08:05.370898 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee4b47a-0c07-4dd1-9482-2481d01b4f48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:08:05.372067 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee4b47a-0c07-4dd1-9482-2481d01b4f48 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:08:05.372362 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee4b47a-0c07-4dd1-9482-2481d01b4f48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:08:05.372645 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee4b47a-0c07-4dd1-9482-2481d01b4f48 admin admin] PATH_INFO: `/v3/users/d9743a442a6444a9aeb1f3b30f4ec10f` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:08:05.374015 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee4b47a-0c07-4dd1-9482-2481d01b4f48 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d9743a442a6444a9aeb1f3b30f4ec10f)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:08:05.378184 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee4b47a-0c07-4dd1-9482-2481d01b4f48 admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:08:05.649899 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69d4f25-b917-4fb5-b77d-6019cf49699c None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:08:05.650882 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69d4f25-b917-4fb5-b77d-6019cf49699c None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:08:05.651374 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69d4f25-b917-4fb5-b77d-6019cf49699c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:08:05.693709 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-e69d4f25-b917-4fb5-b77d-6019cf49699c None None] MFA Rules not processed for user `b334b2401e14419284cce2ad61442d4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:08:05.695455 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-e69d4f25-b917-4fb5-b77d-6019cf49699c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:08:05.711829 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-eee4b47a-0c07-4dd1-9482-2481d01b4f48 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd9743a442a6444a9aeb1f3b30f4ec10f'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:08:05.749978 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-eee4b47a-0c07-4dd1-9482-2481d01b4f48 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd9743a442a6444a9aeb1f3b30f4ec10f'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:08:05.752998 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-eee4b47a-0c07-4dd1-9482-2481d01b4f48 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd9743a442a6444a9aeb1f3b30f4ec10f'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:08:05.827927 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 891/1749] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:08:05 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 10 12:08:05.849437 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-eee4b47a-0c07-4dd1-9482-2481d01b4f48 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd9743a442a6444a9aeb1f3b30f4ec10f'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:08:05.896447 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-eee4b47a-0c07-4dd1-9482-2481d01b4f48 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd9743a442a6444a9aeb1f3b30f4ec10f'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:08:05.924559 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 859/1750] 10.222.0.46 () {58 vars in 1224 bytes} [Mon Oct 10 12:08:05 2022] DELETE /identity/v3/users/d9743a442a6444a9aeb1f3b30f4ec10f => generated 0 bytes in 792 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:08:05.927563 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464b6357-2075-4a81-9eb8-9aeb3c017d8d None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:08:05.928062 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464b6357-2075-4a81-9eb8-9aeb3c017d8d None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:08:05.928499 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464b6357-2075-4a81-9eb8-9aeb3c017d8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:08:05.988282 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-464b6357-2075-4a81-9eb8-9aeb3c017d8d None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:08:05.990707 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-464b6357-2075-4a81-9eb8-9aeb3c017d8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:08:06.228005 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 892/1751] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:08:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 302 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:08:06.240727 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee4b47a-0c07-4dd1-9482-2481d01b4f48 admin admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:08:06.244053 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-eee4b47a-0c07-4dd1-9482-2481d01b4f48 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:08:06.329070 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464b6357-2075-4a81-9eb8-9aeb3c017d8d None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:08:06.330645 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-464b6357-2075-4a81-9eb8-9aeb3c017d8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:08:06.363956 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464b6357-2075-4a81-9eb8-9aeb3c017d8d None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:08:06.379819 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0469a777-c2bd-430f-8944-c0e65f57281d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:08:06.380673 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0469a777-c2bd-430f-8944-c0e65f57281d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:08:06.382262 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0469a777-c2bd-430f-8944-c0e65f57281d admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:08:06.382917 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0469a777-c2bd-430f-8944-c0e65f57281d admin admin] PATH_INFO: `/v3/users/e26432aea0c14ec5ad8ab72935560b7a` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:08:06.384506 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0469a777-c2bd-430f-8944-c0e65f57281d admin admin] RBAC: Authorizing `identity:delete_user(user_id=e26432aea0c14ec5ad8ab72935560b7a)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:08:06.386721 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0469a777-c2bd-430f-8944-c0e65f57281d admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:08:06.485310 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:08:06.514829 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee4b47a-0c07-4dd1-9482-2481d01b4f48 admin admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:08:06.526779 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2425f67d-2248-44cd-a596-8faad2e2913d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58b1c7c85302421cba823ace4201861f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad3772f7b6a47c28b7ba37a9316a3c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:08:06.529941 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2425f67d-2248-44cd-a596-8faad2e2913d service neutron] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:08:06.530228 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2425f67d-2248-44cd-a596-8faad2e2913d service neutron] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:08:06.531943 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2425f67d-2248-44cd-a596-8faad2e2913d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:08:06.572914 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2425f67d-2248-44cd-a596-8faad2e2913d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:08:06.575526 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2425f67d-2248-44cd-a596-8faad2e2913d service neutron] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:08:06.589578 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 860/1752] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 10 12:08:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:08:06.782563 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-0469a777-c2bd-430f-8944-c0e65f57281d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e26432aea0c14ec5ad8ab72935560b7a'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:08:06.799172 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-0469a777-c2bd-430f-8944-c0e65f57281d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e26432aea0c14ec5ad8ab72935560b7a'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:08:06.800514 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-0469a777-c2bd-430f-8944-c0e65f57281d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e26432aea0c14ec5ad8ab72935560b7a'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:08:06.929793 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-0469a777-c2bd-430f-8944-c0e65f57281d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e26432aea0c14ec5ad8ab72935560b7a'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:08:06.963699 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-0469a777-c2bd-430f-8944-c0e65f57281d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e26432aea0c14ec5ad8ab72935560b7a'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:08:06.987725 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec6ec2c-b2fc-43d8-b92a-751d1d01d978 None None] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:08:06.988023 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec6ec2c-b2fc-43d8-b92a-751d1d01d978 None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:08:06.988307 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec6ec2c-b2fc-43d8-b92a-751d1d01d978 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:08:06.992801 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 893/1753] 10.222.0.46 () {58 vars in 1224 bytes} [Mon Oct 10 12:08:06 2022] DELETE /identity/v3/users/e26432aea0c14ec5ad8ab72935560b7a => generated 0 bytes in 654 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:08:07.037125 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.auth.core [None req-bec6ec2c-b2fc-43d8-b92a-751d1d01d978 None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:08:07.038646 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-bec6ec2c-b2fc-43d8-b92a-751d1d01d978 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:08:07.116352 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:08:07.190502 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 861/1754] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:08:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:08:07.203524 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0469a777-c2bd-430f-8944-c0e65f57281d admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:08:07.206097 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-0469a777-c2bd-430f-8944-c0e65f57281d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:08:07.312578 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0469a777-c2bd-430f-8944-c0e65f57281d admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:08:07.317287 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2bd5c51-5a90-4a3f-a220-51b2b8d2c804 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58b1c7c85302421cba823ace4201861f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad3772f7b6a47c28b7ba37a9316a3c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:08:07.318170 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bd5c51-5a90-4a3f-a220-51b2b8d2c804 service neutron] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:08:07.318599 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bd5c51-5a90-4a3f-a220-51b2b8d2c804 service neutron] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:08:07.319060 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bd5c51-5a90-4a3f-a220-51b2b8d2c804 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:08:07.331063 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2bd5c51-5a90-4a3f-a220-51b2b8d2c804 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:08:07.333014 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2bd5c51-5a90-4a3f-a220-51b2b8d2c804 service neutron] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:08:07.350406 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec6ec2c-b2fc-43d8-b92a-751d1d01d978 None None] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:08:07.354415 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-bec6ec2c-b2fc-43d8-b92a-751d1d01d978 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:08:07.361924 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 894/1755] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 10 12:08:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:08:07.381399 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec6ec2c-b2fc-43d8-b92a-751d1d01d978 None None] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:08:07.395410 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87445d2d-023e-4ca7-a378-e2b7797c5dbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:08:07.398814 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87445d2d-023e-4ca7-a378-e2b7797c5dbf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:08:07.401097 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87445d2d-023e-4ca7-a378-e2b7797c5dbf admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:08:07.401916 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87445d2d-023e-4ca7-a378-e2b7797c5dbf admin admin] PATH_INFO: `/v3/projects/2bb99b5b040a48adbf28596e626f2dff` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:08:07.406398 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87445d2d-023e-4ca7-a378-e2b7797c5dbf admin admin] RBAC: Authorizing `identity:delete_project(project_id=2bb99b5b040a48adbf28596e626f2dff)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:08:07.427473 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87445d2d-023e-4ca7-a378-e2b7797c5dbf admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:08:07.664500 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-87445d2d-023e-4ca7-a378-e2b7797c5dbf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2bb99b5b040a48adbf28596e626f2dff', 'request_id': 'req-87445d2d-023e-4ca7-a378-e2b7797c5dbf'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:08:07.688215 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-87445d2d-023e-4ca7-a378-e2b7797c5dbf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2bb99b5b040a48adbf28596e626f2dff', 'request_id': 'req-87445d2d-023e-4ca7-a378-e2b7797c5dbf'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:08:07.702658 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-87445d2d-023e-4ca7-a378-e2b7797c5dbf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2bb99b5b040a48adbf28596e626f2dff', 'request_id': 'req-87445d2d-023e-4ca7-a378-e2b7797c5dbf'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:08:07.815795 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-87445d2d-023e-4ca7-a378-e2b7797c5dbf admin admin] The token cache is being invalidate because project 2bb99b5b040a48adbf28596e626f2dff was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81676) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 10 12:08:07.816092 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-87445d2d-023e-4ca7-a378-e2b7797c5dbf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:08:07.817299 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-87445d2d-023e-4ca7-a378-e2b7797c5dbf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:08:07.820239 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 862/1756] 10.222.0.46 () {58 vars in 1233 bytes} [Mon Oct 10 12:08:07 2022] DELETE /identity/v3/projects/2bb99b5b040a48adbf28596e626f2dff => generated 0 bytes in 470 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:08:07.824623 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2bd5c51-5a90-4a3f-a220-51b2b8d2c804 service neutron] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:08:07.843906 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2bd5c51-5a90-4a3f-a220-51b2b8d2c804 service neutron] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:08:07.975174 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c30416e-7e7c-46fb-91c3-6f8774083f71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:08:07.977220 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c30416e-7e7c-46fb-91c3-6f8774083f71 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:08:07.977678 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c30416e-7e7c-46fb-91c3-6f8774083f71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:08:07.978089 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c30416e-7e7c-46fb-91c3-6f8774083f71 admin admin] PATH_INFO: `/v3/users/1f13f0a9eba346a4bef8634e4d82feab` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:08:07.981338 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c30416e-7e7c-46fb-91c3-6f8774083f71 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1f13f0a9eba346a4bef8634e4d82feab)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:08:07.983693 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c30416e-7e7c-46fb-91c3-6f8774083f71 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:08:08.193938 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87445d2d-023e-4ca7-a378-e2b7797c5dbf admin admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:08:08.197828 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-87445d2d-023e-4ca7-a378-e2b7797c5dbf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:08:08.240915 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87445d2d-023e-4ca7-a378-e2b7797c5dbf admin admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:08:08.255696 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:08:08.264025 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545a42f4-6b77-4bab-ae8d-f6f6fa70a179 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:08:08.265149 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-0c30416e-7e7c-46fb-91c3-6f8774083f71 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1f13f0a9eba346a4bef8634e4d82feab'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:08:08.267484 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545a42f4-6b77-4bab-ae8d-f6f6fa70a179 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:08:08.269895 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545a42f4-6b77-4bab-ae8d-f6f6fa70a179 admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:08:08.271070 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545a42f4-6b77-4bab-ae8d-f6f6fa70a179 admin admin] PATH_INFO: `/v3/projects/94abcb2bea8e40f1ba4d1197f22673aa` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:08:08.277404 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545a42f4-6b77-4bab-ae8d-f6f6fa70a179 admin admin] RBAC: Authorizing `identity:delete_project(project_id=94abcb2bea8e40f1ba4d1197f22673aa)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:08:08.285441 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545a42f4-6b77-4bab-ae8d-f6f6fa70a179 admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:08:08.307255 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-0c30416e-7e7c-46fb-91c3-6f8774083f71 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1f13f0a9eba346a4bef8634e4d82feab'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:08:08.310238 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-0c30416e-7e7c-46fb-91c3-6f8774083f71 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1f13f0a9eba346a4bef8634e4d82feab'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:08:08.447474 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-0c30416e-7e7c-46fb-91c3-6f8774083f71 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1f13f0a9eba346a4bef8634e4d82feab'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:08:08.503524 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-545a42f4-6b77-4bab-ae8d-f6f6fa70a179 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '94abcb2bea8e40f1ba4d1197f22673aa', 'request_id': 'req-545a42f4-6b77-4bab-ae8d-f6f6fa70a179'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:08:08.517423 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-0c30416e-7e7c-46fb-91c3-6f8774083f71 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1f13f0a9eba346a4bef8634e4d82feab'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:08:08.527008 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-545a42f4-6b77-4bab-ae8d-f6f6fa70a179 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '94abcb2bea8e40f1ba4d1197f22673aa', 'request_id': 'req-545a42f4-6b77-4bab-ae8d-f6f6fa70a179'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:08:08.540635 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 895/1757] 10.222.0.46 () {58 vars in 1224 bytes} [Mon Oct 10 12:08:07 2022] DELETE /identity/v3/users/1f13f0a9eba346a4bef8634e4d82feab => generated 0 bytes in 717 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:08:08.560426 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-545a42f4-6b77-4bab-ae8d-f6f6fa70a179 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '94abcb2bea8e40f1ba4d1197f22673aa', 'request_id': 'req-545a42f4-6b77-4bab-ae8d-f6f6fa70a179'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:08:08.713827 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-545a42f4-6b77-4bab-ae8d-f6f6fa70a179 admin admin] The token cache is being invalidate because project 94abcb2bea8e40f1ba4d1197f22673aa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81676) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 10 12:08:08.714386 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-545a42f4-6b77-4bab-ae8d-f6f6fa70a179 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:08:08.715302 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-545a42f4-6b77-4bab-ae8d-f6f6fa70a179 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:08:08.718039 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 863/1758] 10.222.0.46 () {58 vars in 1233 bytes} [Mon Oct 10 12:08:08 2022] DELETE /identity/v3/projects/94abcb2bea8e40f1ba4d1197f22673aa => generated 0 bytes in 525 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 10 12:08:08.722391 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c30416e-7e7c-46fb-91c3-6f8774083f71 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:08:08.724164 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-0c30416e-7e7c-46fb-91c3-6f8774083f71 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:08:08.838314 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545a42f4-6b77-4bab-ae8d-f6f6fa70a179 admin admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:08:08.843526 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-545a42f4-6b77-4bab-ae8d-f6f6fa70a179 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:08:09.057301 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c30416e-7e7c-46fb-91c3-6f8774083f71 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:08:09.067707 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b48b82-767f-41c1-ad6a-e507f878f319 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:08:09.070866 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b48b82-767f-41c1-ad6a-e507f878f319 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:08:09.071467 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b48b82-767f-41c1-ad6a-e507f878f319 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:08:09.071940 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b48b82-767f-41c1-ad6a-e507f878f319 admin admin] PATH_INFO: `/v3/users/d3b878cff9164371a0944d233021265e` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:08:09.076663 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b48b82-767f-41c1-ad6a-e507f878f319 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d3b878cff9164371a0944d233021265e)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:08:09.081372 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b48b82-767f-41c1-ad6a-e507f878f319 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:08:09.145038 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545a42f4-6b77-4bab-ae8d-f6f6fa70a179 admin admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:08:09.153354 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020509b7-5444-4d17-9de6-5786719430d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:08:09.154884 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020509b7-5444-4d17-9de6-5786719430d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:08:09.155343 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020509b7-5444-4d17-9de6-5786719430d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:08:09.155772 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020509b7-5444-4d17-9de6-5786719430d6 admin admin] PATH_INFO: `/v3/users/9c7300856f374d35ba40cb7005dae437` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:08:09.157630 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020509b7-5444-4d17-9de6-5786719430d6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9c7300856f374d35ba40cb7005dae437)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:08:09.161322 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020509b7-5444-4d17-9de6-5786719430d6 admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:08:09.633214 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-05b48b82-767f-41c1-ad6a-e507f878f319 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd3b878cff9164371a0944d233021265e'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:08:09.664411 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:08:09.670945 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-05b48b82-767f-41c1-ad6a-e507f878f319 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd3b878cff9164371a0944d233021265e'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:08:09.670945 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-05b48b82-767f-41c1-ad6a-e507f878f319 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd3b878cff9164371a0944d233021265e'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:08:09.673291 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-020509b7-5444-4d17-9de6-5786719430d6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9c7300856f374d35ba40cb7005dae437'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:08:09.702542 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-020509b7-5444-4d17-9de6-5786719430d6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9c7300856f374d35ba40cb7005dae437'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:08:09.703399 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-020509b7-5444-4d17-9de6-5786719430d6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9c7300856f374d35ba40cb7005dae437'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:08:09.746384 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-05b48b82-767f-41c1-ad6a-e507f878f319 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd3b878cff9164371a0944d233021265e'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:08:09.773402 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-05b48b82-767f-41c1-ad6a-e507f878f319 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd3b878cff9164371a0944d233021265e'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:08:09.825894 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 896/1759] 10.222.0.46 () {58 vars in 1224 bytes} [Mon Oct 10 12:08:08 2022] DELETE /identity/v3/users/d3b878cff9164371a0944d233021265e => generated 0 bytes in 1104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:08:09.837649 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b48b82-767f-41c1-ad6a-e507f878f319 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:08:09.839589 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-05b48b82-767f-41c1-ad6a-e507f878f319 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:08:09.843707 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-020509b7-5444-4d17-9de6-5786719430d6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9c7300856f374d35ba40cb7005dae437'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:08:09.867374 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-020509b7-5444-4d17-9de6-5786719430d6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9c7300856f374d35ba40cb7005dae437'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:08:09.883279 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 864/1760] 10.222.0.46 () {58 vars in 1224 bytes} [Mon Oct 10 12:08:08 2022] DELETE /identity/v3/users/9c7300856f374d35ba40cb7005dae437 => generated 0 bytes in 1046 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:08:09.887910 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c2f518-16c3-41ad-8557-a7a046eb588c None None] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:08:09.888204 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c2f518-16c3-41ad-8557-a7a046eb588c None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:08:09.888480 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c2f518-16c3-41ad-8557-a7a046eb588c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:08:09.962691 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.auth.core [None req-43c2f518-16c3-41ad-8557-a7a046eb588c None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:08:09.970478 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-43c2f518-16c3-41ad-8557-a7a046eb588c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:08:10.036381 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b48b82-767f-41c1-ad6a-e507f878f319 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:08:10.151958 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a7752f-00fa-46b3-b9b0-be076b420dfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:08:10.153457 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a7752f-00fa-46b3-b9b0-be076b420dfb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:08:10.153749 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a7752f-00fa-46b3-b9b0-be076b420dfb admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:08:10.154244 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a7752f-00fa-46b3-b9b0-be076b420dfb admin admin] PATH_INFO: `/v3/projects/da7e6d4dceff4b67a5187f5619cac759` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:08:10.156623 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a7752f-00fa-46b3-b9b0-be076b420dfb admin admin] RBAC: Authorizing `identity:delete_project(project_id=da7e6d4dceff4b67a5187f5619cac759)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:08:10.161356 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a7752f-00fa-46b3-b9b0-be076b420dfb admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:08:10.203670 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 865/1761] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:08:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 317 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:08:10.206638 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9d4923-f00a-44d0-9310-eec594b5e82a None None] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:08:10.212129 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9d4923-f00a-44d0-9310-eec594b5e82a None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:08:10.212422 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9d4923-f00a-44d0-9310-eec594b5e82a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:08:10.405745 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.auth.core [None req-9f9d4923-f00a-44d0-9310-eec594b5e82a None None] MFA Rules not processed for user `d6430593eecf45d8b891c3b55de4a944`. Rule list: `[]` (Enabled: `True`). {{(pid=81676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:08:10.408689 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-9f9d4923-f00a-44d0-9310-eec594b5e82a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:08:10.487887 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-f1a7752f-00fa-46b3-b9b0-be076b420dfb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'da7e6d4dceff4b67a5187f5619cac759', 'request_id': 'req-f1a7752f-00fa-46b3-b9b0-be076b420dfb'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:08:10.503895 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-f1a7752f-00fa-46b3-b9b0-be076b420dfb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'da7e6d4dceff4b67a5187f5619cac759', 'request_id': 'req-f1a7752f-00fa-46b3-b9b0-be076b420dfb'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:08:10.523141 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-f1a7752f-00fa-46b3-b9b0-be076b420dfb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'da7e6d4dceff4b67a5187f5619cac759', 'request_id': 'req-f1a7752f-00fa-46b3-b9b0-be076b420dfb'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:08:10.586151 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 866/1762] 10.222.0.46 () {58 vars in 1009 bytes} [Mon Oct 10 12:08:10 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 382 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:08:10.587316 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9d4923-f00a-44d0-9310-eec594b5e82a None None] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:08:10.589641 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-9f9d4923-f00a-44d0-9310-eec594b5e82a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:08:10.650757 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-f1a7752f-00fa-46b3-b9b0-be076b420dfb admin admin] The token cache is being invalidate because project da7e6d4dceff4b67a5187f5619cac759 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81677) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 10 12:08:10.651104 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-f1a7752f-00fa-46b3-b9b0-be076b420dfb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:08:10.652505 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-f1a7752f-00fa-46b3-b9b0-be076b420dfb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:08:10.655702 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 897/1763] 10.222.0.46 () {58 vars in 1233 bytes} [Mon Oct 10 12:08:09 2022] DELETE /identity/v3/projects/da7e6d4dceff4b67a5187f5619cac759 => generated 0 bytes in 829 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:08:10.659013 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5373c236-d675-4a8f-ac3d-16337f054930 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:08:10.664984 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5373c236-d675-4a8f-ac3d-16337f054930 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:08:10.664984 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5373c236-d675-4a8f-ac3d-16337f054930 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:08:10.740605 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-5373c236-d675-4a8f-ac3d-16337f054930 None None] MFA Rules not processed for user `a7a87488f78444c3ad820527e5fbc099`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:08:10.753260 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-5373c236-d675-4a8f-ac3d-16337f054930 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:08:10.802029 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9d4923-f00a-44d0-9310-eec594b5e82a None None] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:08:10.877619 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 898/1764] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:08:10 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:08:10.879597 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40df66d-f7e3-45dd-bf57-01b1965cc962 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:08:10.879895 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40df66d-f7e3-45dd-bf57-01b1965cc962 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:08:10.880173 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40df66d-f7e3-45dd-bf57-01b1965cc962 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:08:10.898463 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eef2370-cef5-49dd-a1ce-e9cfb2b86d7f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58b1c7c85302421cba823ace4201861f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad3772f7b6a47c28b7ba37a9316a3c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:08:10.900006 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eef2370-cef5-49dd-a1ce-e9cfb2b86d7f service neutron] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:08:10.900449 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eef2370-cef5-49dd-a1ce-e9cfb2b86d7f service neutron] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:08:10.900869 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eef2370-cef5-49dd-a1ce-e9cfb2b86d7f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:08:10.914048 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-0eef2370-cef5-49dd-a1ce-e9cfb2b86d7f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:08:10.915229 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-b40df66d-f7e3-45dd-bf57-01b1965cc962 None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:08:10.916625 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-b40df66d-f7e3-45dd-bf57-01b1965cc962 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:08:11.094801 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 899/1765] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:08:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:08:11.102572 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85046789-16d7-4f19-8282-dea728e251fc None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:08:11.103167 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85046789-16d7-4f19-8282-dea728e251fc None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:08:11.103946 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85046789-16d7-4f19-8282-dea728e251fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:08:11.140540 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eef2370-cef5-49dd-a1ce-e9cfb2b86d7f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:08:11.141851 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-85046789-16d7-4f19-8282-dea728e251fc None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:08:11.142074 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eef2370-cef5-49dd-a1ce-e9cfb2b86d7f service neutron] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:08:11.143331 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-85046789-16d7-4f19-8282-dea728e251fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:08:11.164005 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 867/1766] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 10 12:08:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 577 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:08:11.164917 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eef2370-cef5-49dd-a1ce-e9cfb2b86d7f service neutron] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:08:11.165736 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 900/1767] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:08:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 10 12:08:11.169805 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85046789-16d7-4f19-8282-dea728e251fc None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:08:11.171515 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-0eef2370-cef5-49dd-a1ce-e9cfb2b86d7f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:08:11.190524 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85046789-16d7-4f19-8282-dea728e251fc None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:08:11.203715 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6d234b-e441-4474-ad9f-8f284dfd11dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:08:11.204819 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6d234b-e441-4474-ad9f-8f284dfd11dc admin admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:08:11.205368 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6d234b-e441-4474-ad9f-8f284dfd11dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:08:11.206104 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6d234b-e441-4474-ad9f-8f284dfd11dc admin admin] PATH_INFO: `/v3/domains` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:08:11.207104 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6d234b-e441-4474-ad9f-8f284dfd11dc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 12:08:11.207875 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6d234b-e441-4474-ad9f-8f284dfd11dc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:08:11.209780 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6d234b-e441-4474-ad9f-8f284dfd11dc admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:08:11.248874 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 901/1768] 10.222.0.46 () {58 vars in 1153 bytes} [Mon Oct 10 12:08:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 12:08:11.253283 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6d234b-e441-4474-ad9f-8f284dfd11dc admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:08:11.271149 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6d234b-e441-4474-ad9f-8f284dfd11dc admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:08:11.291146 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a355fe-3b45-4bf4-b22d-be49d0819cf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:08:11.291851 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a355fe-3b45-4bf4-b22d-be49d0819cf1 admin admin] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:08:11.292304 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a355fe-3b45-4bf4-b22d-be49d0819cf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:08:11.292647 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a355fe-3b45-4bf4-b22d-be49d0819cf1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:08:11.294498 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a355fe-3b45-4bf4-b22d-be49d0819cf1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:08:11.300023 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a355fe-3b45-4bf4-b22d-be49d0819cf1 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:08:11.374593 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eef2370-cef5-49dd-a1ce-e9cfb2b86d7f service neutron] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:08:11.381997 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00080181-42ec-450d-9842-d522539b7cbd service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15ef3d53ea484237adb0b3d64fdc5480', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad3772f7b6a47c28b7ba37a9316a3c1', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:08:11.382732 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00080181-42ec-450d-9842-d522539b7cbd service swift] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:08:11.383021 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00080181-42ec-450d-9842-d522539b7cbd service swift] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:08:11.383296 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00080181-42ec-450d-9842-d522539b7cbd service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:08:11.385628 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-00080181-42ec-450d-9842-d522539b7cbd service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:08:11.397925 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:08:11.504396 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 902/1769] 10.222.0.46 () {60 vars in 1153 bytes} [Mon Oct 10 12:08:11 2022] POST /identity/v3/projects => generated 409 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 12:08:11.508814 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a355fe-3b45-4bf4-b22d-be49d0819cf1 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:08:11.613161 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a355fe-3b45-4bf4-b22d-be49d0819cf1 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:08:11.636028 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4adb07-00cf-47ed-948f-18c0db9122ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:08:11.638917 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4adb07-00cf-47ed-948f-18c0db9122ae admin admin] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:08:11.639637 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4adb07-00cf-47ed-948f-18c0db9122ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:08:11.640599 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4adb07-00cf-47ed-948f-18c0db9122ae admin admin] PATH_INFO: `/v3/users` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:08:11.642275 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4adb07-00cf-47ed-948f-18c0db9122ae admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:08:11.646448 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4adb07-00cf-47ed-948f-18c0db9122ae admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:08:11.674566 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00080181-42ec-450d-9842-d522539b7cbd service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:08:11.691388 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00080181-42ec-450d-9842-d522539b7cbd service swift] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:08:11.763568 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 868/1770] 10.222.0.46 () {60 vars in 1384 bytes} [Mon Oct 10 12:08:11 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 599 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 10 12:08:11.774562 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00080181-42ec-450d-9842-d522539b7cbd service swift] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:08:11.791598 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-00080181-42ec-450d-9842-d522539b7cbd service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:08:11.856940 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 903/1771] 10.222.0.46 () {60 vars in 1144 bytes} [Mon Oct 10 12:08:11 2022] POST /identity/v3/users => generated 305 bytes in 349 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 10 12:08:11.858873 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4adb07-00cf-47ed-948f-18c0db9122ae admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:08:11.868904 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00080181-42ec-450d-9842-d522539b7cbd service swift] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:08:11.887934 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c30773-a2aa-47c5-9be6-33ad774da771 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:08:11.887934 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c30773-a2aa-47c5-9be6-33ad774da771 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:08:11.887934 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c30773-a2aa-47c5-9be6-33ad774da771 admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:08:11.887934 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c30773-a2aa-47c5-9be6-33ad774da771 admin admin] PATH_INFO: `/v3/projects/9d8f087abfc148fe985d45c5a92f837b` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:08:11.890184 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c30773-a2aa-47c5-9be6-33ad774da771 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9d8f087abfc148fe985d45c5a92f837b)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:08:11.899834 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c30773-a2aa-47c5-9be6-33ad774da771 admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:08:11.919783 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4adb07-00cf-47ed-948f-18c0db9122ae admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:08:11.951515 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f780ab-f9f2-4c4a-92bf-f95b150cf87a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:08:11.954414 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f780ab-f9f2-4c4a-92bf-f95b150cf87a admin admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:08:11.956535 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f780ab-f9f2-4c4a-92bf-f95b150cf87a admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:08:11.958561 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f780ab-f9f2-4c4a-92bf-f95b150cf87a admin admin] PATH_INFO: `/v3/roles` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:08:11.962917 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f780ab-f9f2-4c4a-92bf-f95b150cf87a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:08:11.972952 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f780ab-f9f2-4c4a-92bf-f95b150cf87a admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:08:12.039285 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 904/1772] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 10 12:08:11 2022] GET /identity/v3/roles => generated 1420 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 12:08:12.043929 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f780ab-f9f2-4c4a-92bf-f95b150cf87a admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:08:12.133653 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f780ab-f9f2-4c4a-92bf-f95b150cf87a admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:08:12.162829 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b805d16-f45f-4a03-892e-71c3d2a3fb0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:08:12.170853 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b805d16-f45f-4a03-892e-71c3d2a3fb0c admin admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:08:12.171829 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b805d16-f45f-4a03-892e-71c3d2a3fb0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:08:12.171829 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b805d16-f45f-4a03-892e-71c3d2a3fb0c admin admin] PATH_INFO: `/v3/roles` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:08:12.177210 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b805d16-f45f-4a03-892e-71c3d2a3fb0c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:08:12.188051 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b805d16-f45f-4a03-892e-71c3d2a3fb0c admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:08:12.263168 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 905/1773] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 10 12:08:12 2022] GET /identity/v3/roles => generated 1420 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 12:08:12.266092 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b805d16-f45f-4a03-892e-71c3d2a3fb0c admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:08:12.320372 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b805d16-f45f-4a03-892e-71c3d2a3fb0c admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:08:12.348386 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-92c30773-a2aa-47c5-9be6-33ad774da771 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9d8f087abfc148fe985d45c5a92f837b', 'request_id': 'req-92c30773-a2aa-47c5-9be6-33ad774da771'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:08:12.384276 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:08:12.413620 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-92c30773-a2aa-47c5-9be6-33ad774da771 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9d8f087abfc148fe985d45c5a92f837b', 'request_id': 'req-92c30773-a2aa-47c5-9be6-33ad774da771'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:08:12.457724 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-92c30773-a2aa-47c5-9be6-33ad774da771 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9d8f087abfc148fe985d45c5a92f837b', 'request_id': 'req-92c30773-a2aa-47c5-9be6-33ad774da771'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:08:12.572362 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c388e67a-1bb4-4b16-85ce-0252378bc460 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:08:12.583706 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c388e67a-1bb4-4b16-85ce-0252378bc460 admin admin] REQUEST_METHOD: `PUT` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:08:12.589492 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c388e67a-1bb4-4b16-85ce-0252378bc460 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:08:12.589492 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c388e67a-1bb4-4b16-85ce-0252378bc460 admin admin] PATH_INFO: `/v3/projects/0a990bbe822443c9b5e465bb6a97e7f8/users/dbfe4e28b2fb46428a20bb0c2fb15c1f/roles/b2cb2a6e818e47338ce071e6fedfc673` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:08:12.631219 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-92c30773-a2aa-47c5-9be6-33ad774da771 admin admin] The token cache is being invalidate because project 9d8f087abfc148fe985d45c5a92f837b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81676) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 10 12:08:12.634999 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-92c30773-a2aa-47c5-9be6-33ad774da771 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:08:12.639353 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-92c30773-a2aa-47c5-9be6-33ad774da771 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:08:12.652243 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 869/1774] 10.222.0.46 () {58 vars in 1233 bytes} [Mon Oct 10 12:08:11 2022] DELETE /identity/v3/projects/9d8f087abfc148fe985d45c5a92f837b => generated 0 bytes in 878 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:08:12.658214 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c388e67a-1bb4-4b16-85ce-0252378bc460 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0a990bbe822443c9b5e465bb6a97e7f8, user_id=dbfe4e28b2fb46428a20bb0c2fb15c1f, role_id=b2cb2a6e818e47338ce071e6fedfc673)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:08:12.660771 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650fcd57-2248-4b37-9ae9-d61ac2547a38 None None] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:08:12.662384 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c388e67a-1bb4-4b16-85ce-0252378bc460 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:08:12.663717 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650fcd57-2248-4b37-9ae9-d61ac2547a38 None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:08:12.669982 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650fcd57-2248-4b37-9ae9-d61ac2547a38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:08:12.742839 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.auth.core [None req-650fcd57-2248-4b37-9ae9-d61ac2547a38 None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:08:12.745346 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-650fcd57-2248-4b37-9ae9-d61ac2547a38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:08:12.778390 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 906/1775] 10.222.0.46 () {60 vars in 1483 bytes} [Mon Oct 10 12:08:12 2022] PUT /identity/v3/projects/0a990bbe822443c9b5e465bb6a97e7f8/users/dbfe4e28b2fb46428a20bb0c2fb15c1f/roles/b2cb2a6e818e47338ce071e6fedfc673 => generated 0 bytes in 515 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:08:12.779588 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c388e67a-1bb4-4b16-85ce-0252378bc460 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:08:12.780167 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 870/1776] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:08:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:08:12.792417 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-c388e67a-1bb4-4b16-85ce-0252378bc460 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:08:12.796352 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25e8fed-1c72-4b40-865b-3067d06c6c5c None None] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:08:12.798096 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25e8fed-1c72-4b40-865b-3067d06c6c5c None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:08:12.798453 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25e8fed-1c72-4b40-865b-3067d06c6c5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:08:12.874439 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.auth.core [None req-a25e8fed-1c72-4b40-865b-3067d06c6c5c None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:08:12.878245 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-a25e8fed-1c72-4b40-865b-3067d06c6c5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:08:13.056562 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c388e67a-1bb4-4b16-85ce-0252378bc460 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:08:13.060315 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 871/1777] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:08:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 265 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:08:13.061811 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18ed8d3-f0b9-4919-890c-fb5da35cf2eb None None] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:08:13.061811 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18ed8d3-f0b9-4919-890c-fb5da35cf2eb None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:08:13.062045 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18ed8d3-f0b9-4919-890c-fb5da35cf2eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:08:13.072037 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:08:13.075480 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd89cfc-e7a6-4a4d-9e03-527cdef62bd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:08:13.078530 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd89cfc-e7a6-4a4d-9e03-527cdef62bd4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:08:13.078931 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd89cfc-e7a6-4a4d-9e03-527cdef62bd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:08:13.079929 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd89cfc-e7a6-4a4d-9e03-527cdef62bd4 admin admin] PATH_INFO: `/v3/projects/90cfbef072a2472a9049df9dcea4964f` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:08:13.085231 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd89cfc-e7a6-4a4d-9e03-527cdef62bd4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=90cfbef072a2472a9049df9dcea4964f)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:08:13.097684 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd89cfc-e7a6-4a4d-9e03-527cdef62bd4 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:08:13.133111 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.auth.core [None req-a18ed8d3-f0b9-4919-890c-fb5da35cf2eb None None] MFA Rules not processed for user `dbfe4e28b2fb46428a20bb0c2fb15c1f`. Rule list: `[]` (Enabled: `True`). {{(pid=81676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:08:13.134874 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-a18ed8d3-f0b9-4919-890c-fb5da35cf2eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:08:13.294522 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-8dd89cfc-e7a6-4a4d-9e03-527cdef62bd4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '90cfbef072a2472a9049df9dcea4964f', 'request_id': 'req-8dd89cfc-e7a6-4a4d-9e03-527cdef62bd4'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:08:13.323712 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-8dd89cfc-e7a6-4a4d-9e03-527cdef62bd4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '90cfbef072a2472a9049df9dcea4964f', 'request_id': 'req-8dd89cfc-e7a6-4a4d-9e03-527cdef62bd4'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:08:13.351782 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-8dd89cfc-e7a6-4a4d-9e03-527cdef62bd4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '90cfbef072a2472a9049df9dcea4964f', 'request_id': 'req-8dd89cfc-e7a6-4a4d-9e03-527cdef62bd4'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:08:13.376873 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 872/1778] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:08:13 2022] POST /identity/v3/auth/tokens => generated 3286 bytes in 316 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:08:13.378865 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18ed8d3-f0b9-4919-890c-fb5da35cf2eb None None] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:08:13.428633 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18ed8d3-f0b9-4919-890c-fb5da35cf2eb None None] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:08:13.441388 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-8dd89cfc-e7a6-4a4d-9e03-527cdef62bd4 admin admin] The token cache is being invalidate because project 90cfbef072a2472a9049df9dcea4964f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81677) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 10 12:08:13.448912 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-8dd89cfc-e7a6-4a4d-9e03-527cdef62bd4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:08:13.450497 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-8dd89cfc-e7a6-4a4d-9e03-527cdef62bd4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:08:13.455903 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 907/1779] 10.222.0.46 () {58 vars in 1233 bytes} [Mon Oct 10 12:08:12 2022] DELETE /identity/v3/projects/90cfbef072a2472a9049df9dcea4964f => generated 0 bytes in 677 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:08:13.461305 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd89cfc-e7a6-4a4d-9e03-527cdef62bd4 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:08:13.463779 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-8dd89cfc-e7a6-4a4d-9e03-527cdef62bd4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:08:13.523210 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:08:13.675227 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd89cfc-e7a6-4a4d-9e03-527cdef62bd4 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:08:13.699126 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cd7da8-272b-43c3-b0ae-608954fe3fb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:08:13.700016 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cd7da8-272b-43c3-b0ae-608954fe3fb0 admin admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:08:13.700486 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cd7da8-272b-43c3-b0ae-608954fe3fb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:08:13.700936 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cd7da8-272b-43c3-b0ae-608954fe3fb0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:08:13.708537 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3cd7da8-272b-43c3-b0ae-608954fe3fb0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 12:08:13.710057 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3cd7da8-272b-43c3-b0ae-608954fe3fb0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:08:13.715837 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0074214e-969b-409d-9599-9153f67f2411 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:08:13.719517 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0074214e-969b-409d-9599-9153f67f2411 admin admin] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:08:13.721541 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0074214e-969b-409d-9599-9153f67f2411 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:08:13.723684 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0074214e-969b-409d-9599-9153f67f2411 admin admin] PATH_INFO: `/v3/projects` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:08:13.730151 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3cd7da8-272b-43c3-b0ae-608954fe3fb0 admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:08:13.770202 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0074214e-969b-409d-9599-9153f67f2411 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:08:13.770202 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0074214e-969b-409d-9599-9153f67f2411 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:08:13.774411 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 873/1780] 10.222.0.46 () {58 vars in 1153 bytes} [Mon Oct 10 12:08:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 397 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 12:08:13.775810 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cd7da8-272b-43c3-b0ae-608954fe3fb0 admin admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:08:13.781500 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-b3cd7da8-272b-43c3-b0ae-608954fe3fb0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:08:13.821364 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cd7da8-272b-43c3-b0ae-608954fe3fb0 admin admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:08:13.836542 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92c29c2-e1fa-48d2-8015-edd3ec68c001 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:08:13.839712 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92c29c2-e1fa-48d2-8015-edd3ec68c001 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:08:13.840335 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92c29c2-e1fa-48d2-8015-edd3ec68c001 admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:08:13.840800 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92c29c2-e1fa-48d2-8015-edd3ec68c001 admin admin] PATH_INFO: `/v3/users/73d94fcd8e614f7699a61177362c18cf` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:08:13.842701 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92c29c2-e1fa-48d2-8015-edd3ec68c001 admin admin] RBAC: Authorizing `identity:delete_user(user_id=73d94fcd8e614f7699a61177362c18cf)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:08:13.890530 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92c29c2-e1fa-48d2-8015-edd3ec68c001 admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:08:14.089290 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:08:14.105607 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 908/1781] 10.222.0.46 () {60 vars in 1153 bytes} [Mon Oct 10 12:08:13 2022] POST /identity/v3/projects => generated 407 bytes in 645 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 12:08:14.107145 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0074214e-969b-409d-9599-9153f67f2411 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:08:14.110023 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-0074214e-969b-409d-9599-9153f67f2411 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:08:14.320550 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-e92c29c2-e1fa-48d2-8015-edd3ec68c001 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '73d94fcd8e614f7699a61177362c18cf'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:08:14.353707 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0074214e-969b-409d-9599-9153f67f2411 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:08:14.354542 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-e92c29c2-e1fa-48d2-8015-edd3ec68c001 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '73d94fcd8e614f7699a61177362c18cf'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:08:14.359293 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-e92c29c2-e1fa-48d2-8015-edd3ec68c001 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '73d94fcd8e614f7699a61177362c18cf'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:08:14.370636 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65438a98-8498-4375-a84d-cfcc5adee1f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:08:14.372140 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65438a98-8498-4375-a84d-cfcc5adee1f7 admin admin] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:08:14.373648 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65438a98-8498-4375-a84d-cfcc5adee1f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:08:14.374318 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65438a98-8498-4375-a84d-cfcc5adee1f7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:08:14.383283 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65438a98-8498-4375-a84d-cfcc5adee1f7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:08:14.390820 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65438a98-8498-4375-a84d-cfcc5adee1f7 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:08:14.492301 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-e92c29c2-e1fa-48d2-8015-edd3ec68c001 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '73d94fcd8e614f7699a61177362c18cf'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:08:14.536861 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-e92c29c2-e1fa-48d2-8015-edd3ec68c001 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '73d94fcd8e614f7699a61177362c18cf'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:08:14.562323 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 909/1782] 10.222.0.46 () {60 vars in 1153 bytes} [Mon Oct 10 12:08:14 2022] POST /identity/v3/projects => generated 373 bytes in 456 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 12:08:14.563696 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65438a98-8498-4375-a84d-cfcc5adee1f7 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:08:14.567443 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-65438a98-8498-4375-a84d-cfcc5adee1f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:08:14.568273 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 874/1783] 10.222.0.46 () {58 vars in 1224 bytes} [Mon Oct 10 12:08:13 2022] DELETE /identity/v3/users/73d94fcd8e614f7699a61177362c18cf => generated 0 bytes in 790 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:08:14.581716 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92c29c2-e1fa-48d2-8015-edd3ec68c001 admin admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:08:14.582554 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-e92c29c2-e1fa-48d2-8015-edd3ec68c001 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:08:14.643853 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:08:14.742141 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65438a98-8498-4375-a84d-cfcc5adee1f7 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:08:14.764566 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950a2967-a648-4298-9ab4-476819342c41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:08:14.765271 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950a2967-a648-4298-9ab4-476819342c41 admin admin] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:08:14.765550 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950a2967-a648-4298-9ab4-476819342c41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:08:14.765823 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950a2967-a648-4298-9ab4-476819342c41 admin admin] PATH_INFO: `/v3/users` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:08:14.766577 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950a2967-a648-4298-9ab4-476819342c41 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:08:14.768457 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950a2967-a648-4298-9ab4-476819342c41 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:08:14.796835 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92c29c2-e1fa-48d2-8015-edd3ec68c001 admin admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:08:14.804774 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbf500e-3766-4b01-a564-b432a4fd60aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:08:14.805849 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbf500e-3766-4b01-a564-b432a4fd60aa admin admin] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:08:14.806284 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbf500e-3766-4b01-a564-b432a4fd60aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:08:14.807158 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbf500e-3766-4b01-a564-b432a4fd60aa admin admin] PATH_INFO: `/v3/users` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:08:14.808432 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dbf500e-3766-4b01-a564-b432a4fd60aa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:08:14.813032 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dbf500e-3766-4b01-a564-b432a4fd60aa admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:08:14.947185 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 910/1784] 10.222.0.46 () {60 vars in 1144 bytes} [Mon Oct 10 12:08:14 2022] POST /identity/v3/users => generated 310 bytes in 384 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 12:08:14.950891 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950a2967-a648-4298-9ab4-476819342c41 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:08:14.964432 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 875/1785] 10.222.0.46 () {60 vars in 1144 bytes} [Mon Oct 10 12:08:14 2022] POST /identity/v3/users => generated 287 bytes in 395 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 12:08:14.966064 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbf500e-3766-4b01-a564-b432a4fd60aa admin admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:08:14.976955 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950a2967-a648-4298-9ab4-476819342c41 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:08:15.010992 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ebb23d-564e-465b-9b07-38a51849d537 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:08:15.017857 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ebb23d-564e-465b-9b07-38a51849d537 admin admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:08:15.018181 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ebb23d-564e-465b-9b07-38a51849d537 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:08:15.018451 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ebb23d-564e-465b-9b07-38a51849d537 admin admin] PATH_INFO: `/v3/roles` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:08:15.019187 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ebb23d-564e-465b-9b07-38a51849d537 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:08:15.024926 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbf500e-3766-4b01-a564-b432a4fd60aa admin admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:08:15.030631 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ebb23d-564e-465b-9b07-38a51849d537 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:08:15.046548 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992b710a-7459-4c15-ba37-33ad56555d98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:08:15.047648 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992b710a-7459-4c15-ba37-33ad56555d98 admin admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:08:15.049473 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992b710a-7459-4c15-ba37-33ad56555d98 admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:08:15.049809 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992b710a-7459-4c15-ba37-33ad56555d98 admin admin] PATH_INFO: `/v3/roles` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:08:15.051084 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992b710a-7459-4c15-ba37-33ad56555d98 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:08:15.056614 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992b710a-7459-4c15-ba37-33ad56555d98 admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:08:15.079078 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 911/1786] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 10 12:08:14 2022] GET /identity/v3/roles => generated 1420 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 10 12:08:15.093783 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ebb23d-564e-465b-9b07-38a51849d537 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:08:15.146365 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 876/1787] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 10 12:08:14 2022] GET /identity/v3/roles => generated 1420 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 12:08:15.152210 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992b710a-7459-4c15-ba37-33ad56555d98 admin admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:08:15.158257 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ebb23d-564e-465b-9b07-38a51849d537 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:08:15.177866 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e336bc63-9b21-4541-b0dd-fab588191a06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:08:15.179717 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e336bc63-9b21-4541-b0dd-fab588191a06 admin admin] REQUEST_METHOD: `PUT` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:08:15.181761 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e336bc63-9b21-4541-b0dd-fab588191a06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:08:15.182381 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e336bc63-9b21-4541-b0dd-fab588191a06 admin admin] PATH_INFO: `/v3/projects/4cb37ff0b2cd4107a5a24f81d7e90fb5/users/7f1d255fdc844eb5893a79309d8601f5/roles/0d822a0674b6495fac623877a3b38025` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:08:15.185996 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992b710a-7459-4c15-ba37-33ad56555d98 admin admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:08:15.204463 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7351ec-0b61-42e3-9d92-1a2a9a070b59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:08:15.206025 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7351ec-0b61-42e3-9d92-1a2a9a070b59 admin admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:08:15.206476 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7351ec-0b61-42e3-9d92-1a2a9a070b59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:08:15.207054 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7351ec-0b61-42e3-9d92-1a2a9a070b59 admin admin] PATH_INFO: `/v3/roles` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:08:15.208296 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7351ec-0b61-42e3-9d92-1a2a9a070b59 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:08:15.211993 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7351ec-0b61-42e3-9d92-1a2a9a070b59 admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:08:15.237124 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e336bc63-9b21-4541-b0dd-fab588191a06 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4cb37ff0b2cd4107a5a24f81d7e90fb5, user_id=7f1d255fdc844eb5893a79309d8601f5, role_id=0d822a0674b6495fac623877a3b38025)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:08:15.254250 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e336bc63-9b21-4541-b0dd-fab588191a06 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:08:15.267312 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 877/1788] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 10 12:08:15 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 12:08:15.267312 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf33543-29e9-4ec9-b8bd-08f9f21ae2fa None None] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:08:15.268124 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf33543-29e9-4ec9-b8bd-08f9f21ae2fa None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:08:15.269774 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf33543-29e9-4ec9-b8bd-08f9f21ae2fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:08:15.309628 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.auth.core [None req-ccf33543-29e9-4ec9-b8bd-08f9f21ae2fa None None] MFA Rules not processed for user `58b1c7c85302421cba823ace4201861f`. Rule list: `[]` (Enabled: `True`). {{(pid=81676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:08:15.312536 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-ccf33543-29e9-4ec9-b8bd-08f9f21ae2fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:08:15.370363 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 912/1789] 10.222.0.46 () {60 vars in 1483 bytes} [Mon Oct 10 12:08:15 2022] PUT /identity/v3/projects/4cb37ff0b2cd4107a5a24f81d7e90fb5/users/7f1d255fdc844eb5893a79309d8601f5/roles/0d822a0674b6495fac623877a3b38025 => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:08:15.373300 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e336bc63-9b21-4541-b0dd-fab588191a06 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:08:15.539729 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 878/1790] 10.222.0.48 () {58 vars in 1015 bytes} [Mon Oct 10 12:08:15 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 274 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:08:15.544398 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf33543-29e9-4ec9-b8bd-08f9f21ae2fa None None] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:08:15.563788 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf33543-29e9-4ec9-b8bd-08f9f21ae2fa None None] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:08:15.633755 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e336bc63-9b21-4541-b0dd-fab588191a06 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:08:15.670256 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ccc26b9-49aa-470e-a4c5-c8b6d762b2b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:08:15.672321 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccc26b9-49aa-470e-a4c5-c8b6d762b2b8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:08:15.674841 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccc26b9-49aa-470e-a4c5-c8b6d762b2b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:08:15.675243 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccc26b9-49aa-470e-a4c5-c8b6d762b2b8 admin admin] PATH_INFO: `/v3/projects/d17d399eda224c16a81052f5faff95b4/users/595183b0f0914211969d5da466dc4598/roles/b2cb2a6e818e47338ce071e6fedfc673` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:08:15.722757 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba58545-942a-4cf9-8ff5-2a2af4058a63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:08:15.725346 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba58545-942a-4cf9-8ff5-2a2af4058a63 admin admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:08:15.728958 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba58545-942a-4cf9-8ff5-2a2af4058a63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:08:15.729138 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ccc26b9-49aa-470e-a4c5-c8b6d762b2b8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d17d399eda224c16a81052f5faff95b4, user_id=595183b0f0914211969d5da466dc4598, role_id=b2cb2a6e818e47338ce071e6fedfc673)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:08:15.729568 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba58545-942a-4cf9-8ff5-2a2af4058a63 admin admin] PATH_INFO: `/v3/roles` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:08:15.731252 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba58545-942a-4cf9-8ff5-2a2af4058a63 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:08:15.736154 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba58545-942a-4cf9-8ff5-2a2af4058a63 admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:08:15.738351 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ccc26b9-49aa-470e-a4c5-c8b6d762b2b8 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:08:15.768623 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 879/1791] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 10 12:08:15 2022] GET /identity/v3/roles => generated 1420 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 12:08:15.773073 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba58545-942a-4cf9-8ff5-2a2af4058a63 admin admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:08:15.775541 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-bba58545-942a-4cf9-8ff5-2a2af4058a63 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:08:15.837293 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 913/1792] 10.222.0.46 () {60 vars in 1483 bytes} [Mon Oct 10 12:08:15 2022] PUT /identity/v3/projects/d17d399eda224c16a81052f5faff95b4/users/595183b0f0914211969d5da466dc4598/roles/b2cb2a6e818e47338ce071e6fedfc673 => generated 0 bytes in 466 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:08:15.845122 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ccc26b9-49aa-470e-a4c5-c8b6d762b2b8 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:08:15.851631 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-3ccc26b9-49aa-470e-a4c5-c8b6d762b2b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:08:15.916605 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba58545-942a-4cf9-8ff5-2a2af4058a63 admin admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:08:16.027273 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ccc26b9-49aa-470e-a4c5-c8b6d762b2b8 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:08:16.031934 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8eadd5e-2baf-478a-a755-6b51babe9335 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58b1c7c85302421cba823ace4201861f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad3772f7b6a47c28b7ba37a9316a3c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:08:16.032801 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8eadd5e-2baf-478a-a755-6b51babe9335 service neutron] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:08:16.033461 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8eadd5e-2baf-478a-a755-6b51babe9335 service neutron] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:08:16.033939 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8eadd5e-2baf-478a-a755-6b51babe9335 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:08:16.046932 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8eadd5e-2baf-478a-a755-6b51babe9335 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:08:16.050908 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8eadd5e-2baf-478a-a755-6b51babe9335 service neutron] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:08:16.066355 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 880/1793] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 10 12:08:15 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:08:16.067672 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8eadd5e-2baf-478a-a755-6b51babe9335 service neutron] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:08:16.071678 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7064f4-02d1-42ec-85a0-f64899b76ba9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58b1c7c85302421cba823ace4201861f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad3772f7b6a47c28b7ba37a9316a3c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:08:16.072467 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7064f4-02d1-42ec-85a0-f64899b76ba9 service neutron] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:08:16.076679 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7064f4-02d1-42ec-85a0-f64899b76ba9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:08:16.078163 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7064f4-02d1-42ec-85a0-f64899b76ba9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:08:16.101396 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7064f4-02d1-42ec-85a0-f64899b76ba9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:08:16.102904 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7064f4-02d1-42ec-85a0-f64899b76ba9 service neutron] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:08:16.125405 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 914/1794] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 10 12:08:15 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 281 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:08:16.127220 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bf20de-cd2a-4817-9898-a9f6eaabc7cf None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:08:16.127666 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bf20de-cd2a-4817-9898-a9f6eaabc7cf None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:08:16.128137 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bf20de-cd2a-4817-9898-a9f6eaabc7cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:08:16.158201 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-68bf20de-cd2a-4817-9898-a9f6eaabc7cf None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:08:16.161259 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-68bf20de-cd2a-4817-9898-a9f6eaabc7cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:08:16.334778 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8eadd5e-2baf-478a-a755-6b51babe9335 service neutron] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:08:16.340606 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2f1ff0-352e-4c78-a98d-9f9fdb0340d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:08:16.343130 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2f1ff0-352e-4c78-a98d-9f9fdb0340d2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:08:16.343527 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2f1ff0-352e-4c78-a98d-9f9fdb0340d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:08:16.343929 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2f1ff0-352e-4c78-a98d-9f9fdb0340d2 admin admin] PATH_INFO: `/v3/projects/4cb37ff0b2cd4107a5a24f81d7e90fb5/users/7f1d255fdc844eb5893a79309d8601f5/roles/0d822a0674b6495fac623877a3b38025` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:08:16.350215 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2f1ff0-352e-4c78-a98d-9f9fdb0340d2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4cb37ff0b2cd4107a5a24f81d7e90fb5, user_id=7f1d255fdc844eb5893a79309d8601f5, role_id=0d822a0674b6495fac623877a3b38025)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:08:16.365098 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2f1ff0-352e-4c78-a98d-9f9fdb0340d2 admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:08:16.393520 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 915/1795] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:08:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 266 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:08:16.401892 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 881/1796] 10.222.0.46 () {60 vars in 1483 bytes} [Mon Oct 10 12:08:16 2022] PUT /identity/v3/projects/4cb37ff0b2cd4107a5a24f81d7e90fb5/users/7f1d255fdc844eb5893a79309d8601f5/roles/0d822a0674b6495fac623877a3b38025 => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:08:16.412350 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69bf57b-630b-4ff1-aed5-38f10ba20e75 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:08:16.412828 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69bf57b-630b-4ff1-aed5-38f10ba20e75 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:08:16.413307 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69bf57b-630b-4ff1-aed5-38f10ba20e75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:08:16.418891 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:08:16.470320 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2f1ff0-352e-4c78-a98d-9f9fdb0340d2 admin admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:08:16.492572 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-a69bf57b-630b-4ff1-aed5-38f10ba20e75 None None] MFA Rules not processed for user `7f1d255fdc844eb5893a79309d8601f5`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:08:16.497936 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-a69bf57b-630b-4ff1-aed5-38f10ba20e75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:08:16.640484 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2f1ff0-352e-4c78-a98d-9f9fdb0340d2 admin admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:08:16.651476 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b59899-7667-410f-af21-c7365a69fff8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58b1c7c85302421cba823ace4201861f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad3772f7b6a47c28b7ba37a9316a3c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:08:16.652413 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b59899-7667-410f-af21-c7365a69fff8 service neutron] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:08:16.653542 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b59899-7667-410f-af21-c7365a69fff8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:08:16.654909 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b59899-7667-410f-af21-c7365a69fff8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:08:16.698424 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 916/1797] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:08:16 2022] POST /identity/v3/auth/tokens => generated 3351 bytes in 287 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:08:16.712441 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69bf57b-630b-4ff1-aed5-38f10ba20e75 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:08:16.719689 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-a69bf57b-630b-4ff1-aed5-38f10ba20e75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:08:16.783125 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:08:16.842493 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b59899-7667-410f-af21-c7365a69fff8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:08:16.845873 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b59899-7667-410f-af21-c7365a69fff8 service neutron] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:08:16.885688 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 882/1798] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 10 12:08:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:08:16.898931 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69bf57b-630b-4ff1-aed5-38f10ba20e75 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:08:16.906713 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e52884-2cb5-4f59-91de-32471a78b268 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3ae093470484b93b23dcfae1657df9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad3772f7b6a47c28b7ba37a9316a3c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:08:16.906713 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e52884-2cb5-4f59-91de-32471a78b268 service cinder] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:08:16.906713 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e52884-2cb5-4f59-91de-32471a78b268 service cinder] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:08:16.907005 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e52884-2cb5-4f59-91de-32471a78b268 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:08:16.924333 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e52884-2cb5-4f59-91de-32471a78b268 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:08:16.932222 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e52884-2cb5-4f59-91de-32471a78b268 service cinder] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:08:16.958279 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 917/1799] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 10 12:08:16 2022] GET /identity/v3/auth/tokens => generated 3351 bytes in 246 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 10 12:08:17.357149 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b59899-7667-410f-af21-c7365a69fff8 service neutron] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:08:17.360688 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-c2b59899-7667-410f-af21-c7365a69fff8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:08:17.402391 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b59899-7667-410f-af21-c7365a69fff8 service neutron] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:08:17.423451 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f456edff-db56-4247-a61c-2e4cbff9ec75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:08:17.424416 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f456edff-db56-4247-a61c-2e4cbff9ec75 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:08:17.424933 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f456edff-db56-4247-a61c-2e4cbff9ec75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:08:17.425362 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f456edff-db56-4247-a61c-2e4cbff9ec75 admin admin] PATH_INFO: `/v3/projects/e67d2ed5531c4042bd67bf241e5ccea4` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:08:17.431111 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f456edff-db56-4247-a61c-2e4cbff9ec75 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e67d2ed5531c4042bd67bf241e5ccea4)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:08:17.438413 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f456edff-db56-4247-a61c-2e4cbff9ec75 admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:08:17.597817 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e52884-2cb5-4f59-91de-32471a78b268 service cinder] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:08:17.611317 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e52884-2cb5-4f59-91de-32471a78b268 service cinder] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:08:17.625282 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-f456edff-db56-4247-a61c-2e4cbff9ec75 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e67d2ed5531c4042bd67bf241e5ccea4', 'request_id': 'req-f456edff-db56-4247-a61c-2e4cbff9ec75'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:08:17.640982 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-f456edff-db56-4247-a61c-2e4cbff9ec75 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e67d2ed5531c4042bd67bf241e5ccea4', 'request_id': 'req-f456edff-db56-4247-a61c-2e4cbff9ec75'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:08:17.659987 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-f456edff-db56-4247-a61c-2e4cbff9ec75 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e67d2ed5531c4042bd67bf241e5ccea4', 'request_id': 'req-f456edff-db56-4247-a61c-2e4cbff9ec75'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:08:17.797007 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d94838f-1520-4ae6-a5ab-14b4240bbec6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3ae093470484b93b23dcfae1657df9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad3772f7b6a47c28b7ba37a9316a3c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:08:17.797586 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d94838f-1520-4ae6-a5ab-14b4240bbec6 service cinder] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:08:17.798493 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d94838f-1520-4ae6-a5ab-14b4240bbec6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:08:17.798809 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d94838f-1520-4ae6-a5ab-14b4240bbec6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:08:17.807068 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-0d94838f-1520-4ae6-a5ab-14b4240bbec6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:08:17.814661 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-f456edff-db56-4247-a61c-2e4cbff9ec75 admin admin] The token cache is being invalidate because project e67d2ed5531c4042bd67bf241e5ccea4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81676) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 10 12:08:17.820583 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-f456edff-db56-4247-a61c-2e4cbff9ec75 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:08:17.820583 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-f456edff-db56-4247-a61c-2e4cbff9ec75 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:08:17.821895 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 883/1800] 10.222.0.46 () {58 vars in 1233 bytes} [Mon Oct 10 12:08:17 2022] DELETE /identity/v3/projects/e67d2ed5531c4042bd67bf241e5ccea4 => generated 0 bytes in 466 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:08:17.848588 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4850c27d-ec5b-41b5-bd2a-92b0c0d9bcdb None None] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:08:17.852006 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4850c27d-ec5b-41b5-bd2a-92b0c0d9bcdb None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:08:17.852318 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4850c27d-ec5b-41b5-bd2a-92b0c0d9bcdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:08:17.910767 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.auth.core [None req-4850c27d-ec5b-41b5-bd2a-92b0c0d9bcdb None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:08:17.918162 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-4850c27d-ec5b-41b5-bd2a-92b0c0d9bcdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:08:18.005965 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d94838f-1520-4ae6-a5ab-14b4240bbec6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:08:18.013588 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d94838f-1520-4ae6-a5ab-14b4240bbec6 service cinder] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:08:18.024114 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:08:18.042165 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 918/1801] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 10 12:08:17 2022] GET /identity/v3/auth/tokens => generated 3286 bytes in 445 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:08:18.046437 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 884/1802] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:08:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 10 12:08:18.054582 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aaa6766-9aa9-47a1-9227-c4eb990cdc9a None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:08:18.054981 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aaa6766-9aa9-47a1-9227-c4eb990cdc9a None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:08:18.055346 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aaa6766-9aa9-47a1-9227-c4eb990cdc9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:08:18.122086 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-5aaa6766-9aa9-47a1-9227-c4eb990cdc9a None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:08:18.124631 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-5aaa6766-9aa9-47a1-9227-c4eb990cdc9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:08:18.137402 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 919/1803] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:08:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:08:18.140474 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4850c27d-ec5b-41b5-bd2a-92b0c0d9bcdb None None] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:08:18.173321 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4850c27d-ec5b-41b5-bd2a-92b0c0d9bcdb None None] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:08:18.183861 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3a4db4-b8bb-44b8-b9a6-b09b137d302d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:08:18.184625 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3a4db4-b8bb-44b8-b9a6-b09b137d302d admin admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:08:18.185053 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3a4db4-b8bb-44b8-b9a6-b09b137d302d admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:08:18.185519 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3a4db4-b8bb-44b8-b9a6-b09b137d302d admin admin] PATH_INFO: `/v3/domains` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:08:18.187178 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3a4db4-b8bb-44b8-b9a6-b09b137d302d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 12:08:18.187471 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3a4db4-b8bb-44b8-b9a6-b09b137d302d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:08:18.189225 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3a4db4-b8bb-44b8-b9a6-b09b137d302d admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:08:18.231061 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 885/1804] 10.222.0.46 () {58 vars in 1153 bytes} [Mon Oct 10 12:08:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 12:08:18.236530 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aaa6766-9aa9-47a1-9227-c4eb990cdc9a None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:08:18.251560 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aaa6766-9aa9-47a1-9227-c4eb990cdc9a None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:08:18.259959 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613090e0-722c-4371-a811-c8846f0641b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:08:18.260918 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613090e0-722c-4371-a811-c8846f0641b8 admin admin] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:08:18.261253 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613090e0-722c-4371-a811-c8846f0641b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:08:18.261616 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613090e0-722c-4371-a811-c8846f0641b8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:08:18.262383 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613090e0-722c-4371-a811-c8846f0641b8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:08:18.266272 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613090e0-722c-4371-a811-c8846f0641b8 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:08:18.437401 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 920/1805] 10.222.0.46 () {60 vars in 1153 bytes} [Mon Oct 10 12:08:18 2022] POST /identity/v3/projects => generated 399 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 12:08:18.444914 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3a4db4-b8bb-44b8-b9a6-b09b137d302d admin admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:08:18.510471 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3a4db4-b8bb-44b8-b9a6-b09b137d302d admin admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:08:18.577682 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:08:18.660619 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72dae3c-ebfb-4fcc-b35c-8ebf8526f62d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:08:18.661247 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72dae3c-ebfb-4fcc-b35c-8ebf8526f62d admin admin] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:08:18.661824 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72dae3c-ebfb-4fcc-b35c-8ebf8526f62d admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:08:18.662247 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72dae3c-ebfb-4fcc-b35c-8ebf8526f62d admin admin] PATH_INFO: `/v3/users` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:08:18.663116 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72dae3c-ebfb-4fcc-b35c-8ebf8526f62d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:08:18.681263 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72dae3c-ebfb-4fcc-b35c-8ebf8526f62d admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:08:18.848068 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 886/1806] 10.222.0.46 () {60 vars in 1144 bytes} [Mon Oct 10 12:08:18 2022] POST /identity/v3/users => generated 300 bytes in 407 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 12:08:18.853321 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613090e0-722c-4371-a811-c8846f0641b8 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:08:18.877603 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613090e0-722c-4371-a811-c8846f0641b8 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:08:18.884410 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df04284-8621-4fad-a629-275d0a51a99e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:08:18.886109 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df04284-8621-4fad-a629-275d0a51a99e admin admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:08:18.886475 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df04284-8621-4fad-a629-275d0a51a99e admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:08:18.887385 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df04284-8621-4fad-a629-275d0a51a99e admin admin] PATH_INFO: `/v3/roles` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:08:18.888455 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df04284-8621-4fad-a629-275d0a51a99e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:08:18.891907 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df04284-8621-4fad-a629-275d0a51a99e admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:08:18.935637 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 921/1807] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 10 12:08:18 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 12:08:18.939967 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72dae3c-ebfb-4fcc-b35c-8ebf8526f62d admin admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:08:18.958058 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72dae3c-ebfb-4fcc-b35c-8ebf8526f62d admin admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:08:18.965826 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf08da19-0535-4662-98ee-47e8025b2f74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:08:18.983554 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf08da19-0535-4662-98ee-47e8025b2f74 admin admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:08:18.986395 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf08da19-0535-4662-98ee-47e8025b2f74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:08:18.986395 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf08da19-0535-4662-98ee-47e8025b2f74 admin admin] PATH_INFO: `/v3/roles` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:08:18.986395 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf08da19-0535-4662-98ee-47e8025b2f74 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:08:18.988432 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf08da19-0535-4662-98ee-47e8025b2f74 admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:08:19.004363 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 887/1808] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 10 12:08:18 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 12:08:19.012087 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df04284-8621-4fad-a629-275d0a51a99e admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:08:19.053863 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df04284-8621-4fad-a629-275d0a51a99e admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:08:19.081796 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a1fbcf-bc11-4561-b60b-46b9c43e0426 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:08:19.086564 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a1fbcf-bc11-4561-b60b-46b9c43e0426 admin admin] REQUEST_METHOD: `PUT` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:08:19.089848 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a1fbcf-bc11-4561-b60b-46b9c43e0426 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:08:19.093088 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a1fbcf-bc11-4561-b60b-46b9c43e0426 admin admin] PATH_INFO: `/v3/projects/ec6ecce071ae4b72a1ee50e780f65112/users/47ca6459e866442e94df1cb09e6cc5a0/roles/b2cb2a6e818e47338ce071e6fedfc673` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:08:19.221260 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a1fbcf-bc11-4561-b60b-46b9c43e0426 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ec6ecce071ae4b72a1ee50e780f65112, user_id=47ca6459e866442e94df1cb09e6cc5a0, role_id=b2cb2a6e818e47338ce071e6fedfc673)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:08:19.284229 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a1fbcf-bc11-4561-b60b-46b9c43e0426 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:08:19.446653 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 922/1809] 10.222.0.46 () {60 vars in 1483 bytes} [Mon Oct 10 12:08:19 2022] PUT /identity/v3/projects/ec6ecce071ae4b72a1ee50e780f65112/users/47ca6459e866442e94df1cb09e6cc5a0/roles/b2cb2a6e818e47338ce071e6fedfc673 => generated 0 bytes in 435 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:08:19.466649 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:08:19.472526 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742e7d94-6254-475f-961d-a727e4517097 None None] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:08:19.475038 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742e7d94-6254-475f-961d-a727e4517097 None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:08:19.475038 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742e7d94-6254-475f-961d-a727e4517097 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:08:19.579738 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.auth.core [None req-742e7d94-6254-475f-961d-a727e4517097 None None] MFA Rules not processed for user `47ca6459e866442e94df1cb09e6cc5a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:08:19.582091 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-742e7d94-6254-475f-961d-a727e4517097 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:08:19.822132 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 888/1810] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:08:19 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 352 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:08:19.829397 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a1fbcf-bc11-4561-b60b-46b9c43e0426 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:08:20.091655 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a1fbcf-bc11-4561-b60b-46b9c43e0426 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:08:20.142000 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bca74ed-0176-4650-a509-cb9377c0f734 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:08:20.147771 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bca74ed-0176-4650-a509-cb9377c0f734 admin admin] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:08:20.148414 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bca74ed-0176-4650-a509-cb9377c0f734 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:08:20.152124 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bca74ed-0176-4650-a509-cb9377c0f734 admin admin] PATH_INFO: `/v3/projects` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:08:20.152124 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bca74ed-0176-4650-a509-cb9377c0f734 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:08:20.157773 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bca74ed-0176-4650-a509-cb9377c0f734 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:08:20.359530 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 923/1811] 10.222.0.46 () {60 vars in 1153 bytes} [Mon Oct 10 12:08:19 2022] POST /identity/v3/projects => generated 399 bytes in 532 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 12:08:20.366100 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742e7d94-6254-475f-961d-a727e4517097 None None] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:08:20.436803 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742e7d94-6254-475f-961d-a727e4517097 None None] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:08:20.581415 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3a4c79-f236-48b9-875b-cd06212039b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:08:20.595374 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3a4c79-f236-48b9-875b-cd06212039b4 admin admin] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:08:20.595631 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3a4c79-f236-48b9-875b-cd06212039b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:08:20.596086 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3a4c79-f236-48b9-875b-cd06212039b4 admin admin] PATH_INFO: `/v3/users` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:08:20.596940 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3a4c79-f236-48b9-875b-cd06212039b4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:08:20.601235 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3a4c79-f236-48b9-875b-cd06212039b4 admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:08:20.689625 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:08:20.699426 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 889/1812] 10.222.0.46 () {60 vars in 1144 bytes} [Mon Oct 10 12:08:20 2022] POST /identity/v3/users => generated 306 bytes in 335 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 12:08:20.707717 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bca74ed-0176-4650-a509-cb9377c0f734 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:08:20.734967 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bca74ed-0176-4650-a509-cb9377c0f734 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:08:20.750322 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71af5fa-9371-4bac-b37c-933e74fa3bb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:08:20.753055 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71af5fa-9371-4bac-b37c-933e74fa3bb7 admin admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:08:20.753055 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71af5fa-9371-4bac-b37c-933e74fa3bb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:08:20.753055 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71af5fa-9371-4bac-b37c-933e74fa3bb7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:08:20.753055 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71af5fa-9371-4bac-b37c-933e74fa3bb7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:08:20.754559 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71af5fa-9371-4bac-b37c-933e74fa3bb7 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:08:20.755602 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54c2ecc-38ca-446c-a0e0-04d4037b9feb None None] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:08:20.756066 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54c2ecc-38ca-446c-a0e0-04d4037b9feb None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:08:20.756604 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54c2ecc-38ca-446c-a0e0-04d4037b9feb None None] PATH_INFO: `/` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:08:20.758724 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 890/1813] 10.222.0.46 () {54 vars in 921 bytes} [Mon Oct 10 12:08:20 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 12:08:20.783630 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8f738a-4cc4-4081-9cb4-0b5705656c4e None None] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:08:20.785490 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8f738a-4cc4-4081-9cb4-0b5705656c4e None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:08:20.785490 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8f738a-4cc4-4081-9cb4-0b5705656c4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:08:20.801679 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 924/1814] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 10 12:08:20 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 10 12:08:20.816262 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71af5fa-9371-4bac-b37c-933e74fa3bb7 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:08:20.828989 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.auth.core [None req-1d8f738a-4cc4-4081-9cb4-0b5705656c4e None None] MFA Rules not processed for user `5f130bbb69e546ce99b4e14de81d7e54`. Rule list: `[]` (Enabled: `True`). {{(pid=81676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:08:20.832551 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71af5fa-9371-4bac-b37c-933e74fa3bb7 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:08:20.833344 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-1d8f738a-4cc4-4081-9cb4-0b5705656c4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:08:20.839168 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdce3d3f-45cb-437c-b4d5-d133b44f5f11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:08:20.842872 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdce3d3f-45cb-437c-b4d5-d133b44f5f11 admin admin] REQUEST_METHOD: `PUT` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:08:20.842872 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdce3d3f-45cb-437c-b4d5-d133b44f5f11 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:08:20.842872 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdce3d3f-45cb-437c-b4d5-d133b44f5f11 admin admin] PATH_INFO: `/v3/projects/04b64d7d543a4c24892f9d1747a94735/users/5f8ca4578e29466586f011aeafefc1e2/roles/0d822a0674b6495fac623877a3b38025` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:08:20.848277 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:08:20.895190 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdce3d3f-45cb-437c-b4d5-d133b44f5f11 admin admin] RBAC: Authorizing `identity:create_grant(project_id=04b64d7d543a4c24892f9d1747a94735, user_id=5f8ca4578e29466586f011aeafefc1e2, role_id=0d822a0674b6495fac623877a3b38025)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:08:20.905778 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdce3d3f-45cb-437c-b4d5-d133b44f5f11 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:08:21.037506 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 891/1815] 10.222.0.46 () {58 vars in 1018 bytes} [Mon Oct 10 12:08:20 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 10 12:08:21.083519 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 925/1816] 10.222.0.46 () {60 vars in 1483 bytes} [Mon Oct 10 12:08:20 2022] PUT /identity/v3/projects/04b64d7d543a4c24892f9d1747a94735/users/5f8ca4578e29466586f011aeafefc1e2/roles/0d822a0674b6495fac623877a3b38025 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:08:21.089451 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8f738a-4cc4-4081-9cb4-0b5705656c4e None None] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:08:21.129327 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8f738a-4cc4-4081-9cb4-0b5705656c4e None None] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:08:21.241986 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96117d9c-8e97-4971-a80e-0212fa66e1f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:08:21.243484 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96117d9c-8e97-4971-a80e-0212fa66e1f9 admin admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:08:21.244337 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96117d9c-8e97-4971-a80e-0212fa66e1f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:08:21.244877 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96117d9c-8e97-4971-a80e-0212fa66e1f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:08:21.245975 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96117d9c-8e97-4971-a80e-0212fa66e1f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:08:21.249481 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96117d9c-8e97-4971-a80e-0212fa66e1f9 admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:08:21.278854 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdce3d3f-45cb-437c-b4d5-d133b44f5f11 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:08:21.282059 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 892/1817] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 10 12:08:21 2022] GET /identity/v3/roles => generated 1420 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 12:08:21.299389 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdce3d3f-45cb-437c-b4d5-d133b44f5f11 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:08:21.311025 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3d2a20-ee00-405d-8e5a-e4cfb588fbd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:08:21.312850 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3d2a20-ee00-405d-8e5a-e4cfb588fbd1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:08:21.312985 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3d2a20-ee00-405d-8e5a-e4cfb588fbd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:08:21.313410 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3d2a20-ee00-405d-8e5a-e4cfb588fbd1 admin admin] PATH_INFO: `/v3/projects/04b64d7d543a4c24892f9d1747a94735/users/5f8ca4578e29466586f011aeafefc1e2/roles/0d822a0674b6495fac623877a3b38025` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:08:21.317749 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3d2a20-ee00-405d-8e5a-e4cfb588fbd1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=04b64d7d543a4c24892f9d1747a94735, user_id=5f8ca4578e29466586f011aeafefc1e2, role_id=0d822a0674b6495fac623877a3b38025)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:08:21.324784 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3d2a20-ee00-405d-8e5a-e4cfb588fbd1 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:08:21.346816 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 926/1818] 10.222.0.46 () {60 vars in 1483 bytes} [Mon Oct 10 12:08:21 2022] PUT /identity/v3/projects/04b64d7d543a4c24892f9d1747a94735/users/5f8ca4578e29466586f011aeafefc1e2/roles/0d822a0674b6495fac623877a3b38025 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 10 12:08:21.367481 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6734a479-6ac9-4d22-9e52-9be3a214bc7a None None] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:08:21.367946 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6734a479-6ac9-4d22-9e52-9be3a214bc7a None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:08:21.368618 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6734a479-6ac9-4d22-9e52-9be3a214bc7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:08:21.470173 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.auth.core [None req-6734a479-6ac9-4d22-9e52-9be3a214bc7a None None] MFA Rules not processed for user `5f8ca4578e29466586f011aeafefc1e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:08:21.475044 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-6734a479-6ac9-4d22-9e52-9be3a214bc7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:08:21.614890 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 893/1819] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:08:21 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 10 12:08:21.634779 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3d2a20-ee00-405d-8e5a-e4cfb588fbd1 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:08:21.638225 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-db3d2a20-ee00-405d-8e5a-e4cfb588fbd1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:08:21.662785 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7ec2b4-11b1-4174-9be7-859e8e673d96 None None] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:08:21.663642 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7ec2b4-11b1-4174-9be7-859e8e673d96 None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:08:21.663804 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7ec2b4-11b1-4174-9be7-859e8e673d96 None None] PATH_INFO: `/` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:08:21.665586 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 894/1820] 10.222.0.46 () {54 vars in 921 bytes} [Mon Oct 10 12:08:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 12:08:21.768449 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9288df6-a426-4f79-95eb-b5f4610d3acb None None] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:08:21.771355 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9288df6-a426-4f79-95eb-b5f4610d3acb None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:08:21.775270 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9288df6-a426-4f79-95eb-b5f4610d3acb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:08:21.844785 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.auth.core [None req-e9288df6-a426-4f79-95eb-b5f4610d3acb None None] MFA Rules not processed for user `5f130bbb69e546ce99b4e14de81d7e54`. Rule list: `[]` (Enabled: `True`). {{(pid=81676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:08:21.847164 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-e9288df6-a426-4f79-95eb-b5f4610d3acb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:08:21.850874 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3d2a20-ee00-405d-8e5a-e4cfb588fbd1 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:08:21.882736 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011cf29c-bce2-48d2-a03c-76bb92e665bd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3ae093470484b93b23dcfae1657df9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad3772f7b6a47c28b7ba37a9316a3c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:08:21.884158 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011cf29c-bce2-48d2-a03c-76bb92e665bd service cinder] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:08:21.885003 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011cf29c-bce2-48d2-a03c-76bb92e665bd service cinder] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:08:21.886476 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011cf29c-bce2-48d2-a03c-76bb92e665bd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:08:21.911154 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011cf29c-bce2-48d2-a03c-76bb92e665bd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:08:21.915552 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011cf29c-bce2-48d2-a03c-76bb92e665bd service cinder] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:08:21.966823 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 927/1821] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 10 12:08:21 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:08:21.988638 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 895/1822] 10.222.0.46 () {58 vars in 1018 bytes} [Mon Oct 10 12:08:21 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:08:22.012327 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:08:22.317026 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011cf29c-bce2-48d2-a03c-76bb92e665bd service cinder] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:08:22.360597 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011cf29c-bce2-48d2-a03c-76bb92e665bd service cinder] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:08:22.375297 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f33a70-cb8a-46b3-96f7-e41b1167a1e0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3ae093470484b93b23dcfae1657df9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad3772f7b6a47c28b7ba37a9316a3c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:08:22.377261 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f33a70-cb8a-46b3-96f7-e41b1167a1e0 service cinder] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:08:22.377261 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f33a70-cb8a-46b3-96f7-e41b1167a1e0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:08:22.377572 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f33a70-cb8a-46b3-96f7-e41b1167a1e0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:08:22.420445 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:08:22.614097 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f33a70-cb8a-46b3-96f7-e41b1167a1e0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:08:22.619893 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f33a70-cb8a-46b3-96f7-e41b1167a1e0 service cinder] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:08:22.632849 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 928/1823] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 10 12:08:22 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:08:22.923299 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9288df6-a426-4f79-95eb-b5f4610d3acb None None] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:08:22.925359 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-e9288df6-a426-4f79-95eb-b5f4610d3acb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:08:22.961054 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9288df6-a426-4f79-95eb-b5f4610d3acb None None] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:08:22.980939 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d15af3-2688-4c1e-89e0-103bbd923e94 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f130bbb69e546ce99b4e14de81d7e54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad3772f7b6a47c28b7ba37a9316a3c1', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:08:22.981589 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d15af3-2688-4c1e-89e0-103bbd923e94 service nova] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:08:22.981875 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d15af3-2688-4c1e-89e0-103bbd923e94 service nova] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:08:22.982177 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d15af3-2688-4c1e-89e0-103bbd923e94 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:08:23.016803 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d15af3-2688-4c1e-89e0-103bbd923e94 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:08:23.020986 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d15af3-2688-4c1e-89e0-103bbd923e94 service nova] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:08:23.099374 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 896/1824] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 10 12:08:22 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 10 12:08:23.714856 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f33a70-cb8a-46b3-96f7-e41b1167a1e0 service cinder] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:08:23.717510 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-c2f33a70-cb8a-46b3-96f7-e41b1167a1e0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:08:23.777497 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f33a70-cb8a-46b3-96f7-e41b1167a1e0 service cinder] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:08:23.794549 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62521974-aeeb-4f9a-b29b-ec9884f73554 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f130bbb69e546ce99b4e14de81d7e54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad3772f7b6a47c28b7ba37a9316a3c1', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:08:23.796477 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62521974-aeeb-4f9a-b29b-ec9884f73554 service nova] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:08:23.799540 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62521974-aeeb-4f9a-b29b-ec9884f73554 service nova] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:08:23.799540 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62521974-aeeb-4f9a-b29b-ec9884f73554 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:08:23.849426 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62521974-aeeb-4f9a-b29b-ec9884f73554 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:08:23.857252 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62521974-aeeb-4f9a-b29b-ec9884f73554 service nova] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:08:23.889389 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 929/1825] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 10 12:08:23 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:08:24.134418 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:08:24.292086 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed0ff06-195f-408e-8596-5114c2af4e10 None None] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:08:24.294288 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed0ff06-195f-408e-8596-5114c2af4e10 None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:08:24.294819 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed0ff06-195f-408e-8596-5114c2af4e10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:08:24.379972 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.auth.core [None req-1ed0ff06-195f-408e-8596-5114c2af4e10 None None] MFA Rules not processed for user `595183b0f0914211969d5da466dc4598`. Rule list: `[]` (Enabled: `True`). {{(pid=81676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:08:24.382903 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-1ed0ff06-195f-408e-8596-5114c2af4e10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:08:24.541448 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 897/1826] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:08:24 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:08:24.548414 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62521974-aeeb-4f9a-b29b-ec9884f73554 service nova] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:08:24.551579 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-62521974-aeeb-4f9a-b29b-ec9884f73554 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:08:24.661862 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62521974-aeeb-4f9a-b29b-ec9884f73554 service nova] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:08:24.666532 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b23213-2b56-436b-a929-f294387810fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:08:24.667227 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b23213-2b56-436b-a929-f294387810fd admin admin] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:08:24.667509 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b23213-2b56-436b-a929-f294387810fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:08:24.667787 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b23213-2b56-436b-a929-f294387810fd admin admin] PATH_INFO: `/v3/projects` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:08:24.668431 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b23213-2b56-436b-a929-f294387810fd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:08:24.670416 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b23213-2b56-436b-a929-f294387810fd admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:08:24.774716 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 930/1827] 10.222.0.46 () {60 vars in 1153 bytes} [Mon Oct 10 12:08:24 2022] POST /identity/v3/projects => generated 371 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 12:08:24.775698 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed0ff06-195f-408e-8596-5114c2af4e10 None None] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:08:24.795968 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed0ff06-195f-408e-8596-5114c2af4e10 None None] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:08:24.833440 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:08:24.891522 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183ae556-2e57-4698-9fc5-78077ef5d058 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:08:24.892284 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183ae556-2e57-4698-9fc5-78077ef5d058 admin admin] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:08:24.892748 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183ae556-2e57-4698-9fc5-78077ef5d058 admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:08:24.893358 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183ae556-2e57-4698-9fc5-78077ef5d058 admin admin] PATH_INFO: `/v3/users` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:08:24.894165 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183ae556-2e57-4698-9fc5-78077ef5d058 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:08:24.896868 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183ae556-2e57-4698-9fc5-78077ef5d058 admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:08:25.020487 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 898/1828] 10.222.0.46 () {60 vars in 1144 bytes} [Mon Oct 10 12:08:24 2022] POST /identity/v3/users => generated 292 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 12:08:25.025225 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b23213-2b56-436b-a929-f294387810fd admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:08:25.038114 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b23213-2b56-436b-a929-f294387810fd admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:08:25.045868 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01fc109-c2a2-418d-909d-fc14faca9cc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:08:25.046819 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01fc109-c2a2-418d-909d-fc14faca9cc5 admin admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:08:25.047127 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01fc109-c2a2-418d-909d-fc14faca9cc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:08:25.050712 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01fc109-c2a2-418d-909d-fc14faca9cc5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:08:25.050712 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01fc109-c2a2-418d-909d-fc14faca9cc5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:08:25.050712 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01fc109-c2a2-418d-909d-fc14faca9cc5 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:08:25.065862 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 931/1829] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 10 12:08:25 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 10 12:08:25.067208 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183ae556-2e57-4698-9fc5-78077ef5d058 admin admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:08:25.078258 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183ae556-2e57-4698-9fc5-78077ef5d058 admin admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:08:25.084454 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c922fad5-35b9-4c3e-88d5-443e8448d97f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:08:25.085146 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c922fad5-35b9-4c3e-88d5-443e8448d97f admin admin] REQUEST_METHOD: `PUT` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:08:25.085431 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c922fad5-35b9-4c3e-88d5-443e8448d97f admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:08:25.085706 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c922fad5-35b9-4c3e-88d5-443e8448d97f admin admin] PATH_INFO: `/v3/projects/45f32ff86e37403abcda138fa40c8374/users/0886c8e7e1814e51a5b9b20807d33098/roles/0d822a0674b6495fac623877a3b38025` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:08:25.105251 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c922fad5-35b9-4c3e-88d5-443e8448d97f admin admin] RBAC: Authorizing `identity:create_grant(project_id=45f32ff86e37403abcda138fa40c8374, user_id=0886c8e7e1814e51a5b9b20807d33098, role_id=0d822a0674b6495fac623877a3b38025)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:08:25.110069 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c922fad5-35b9-4c3e-88d5-443e8448d97f admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:08:25.122063 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:08:25.226359 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 899/1830] 10.222.0.46 () {60 vars in 1483 bytes} [Mon Oct 10 12:08:25 2022] PUT /identity/v3/projects/45f32ff86e37403abcda138fa40c8374/users/0886c8e7e1814e51a5b9b20807d33098/roles/0d822a0674b6495fac623877a3b38025 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:08:25.230987 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01fc109-c2a2-418d-909d-fc14faca9cc5 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:08:25.242224 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01fc109-c2a2-418d-909d-fc14faca9cc5 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:08:25.307864 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8288729a-f52f-4118-8128-336f4eb6320c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:08:25.309746 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8288729a-f52f-4118-8128-336f4eb6320c admin admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:08:25.311333 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8288729a-f52f-4118-8128-336f4eb6320c admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:08:25.311639 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8288729a-f52f-4118-8128-336f4eb6320c admin admin] PATH_INFO: `/v3/roles` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:08:25.312313 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8288729a-f52f-4118-8128-336f4eb6320c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:08:25.313920 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8288729a-f52f-4118-8128-336f4eb6320c admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:08:25.326181 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 932/1831] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 10 12:08:25 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 10 12:08:25.333614 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c922fad5-35b9-4c3e-88d5-443e8448d97f admin admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:08:25.345365 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c922fad5-35b9-4c3e-88d5-443e8448d97f admin admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:08:25.351880 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3d58c6-3010-487f-bccc-56e196e6156b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:08:25.352582 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3d58c6-3010-487f-bccc-56e196e6156b admin admin] REQUEST_METHOD: `PUT` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:08:25.352867 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3d58c6-3010-487f-bccc-56e196e6156b admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:08:25.353439 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3d58c6-3010-487f-bccc-56e196e6156b admin admin] PATH_INFO: `/v3/projects/45f32ff86e37403abcda138fa40c8374/users/0886c8e7e1814e51a5b9b20807d33098/roles/0d822a0674b6495fac623877a3b38025` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:08:25.355478 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3d58c6-3010-487f-bccc-56e196e6156b admin admin] RBAC: Authorizing `identity:create_grant(project_id=45f32ff86e37403abcda138fa40c8374, user_id=0886c8e7e1814e51a5b9b20807d33098, role_id=0d822a0674b6495fac623877a3b38025)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:08:25.358981 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3d58c6-3010-487f-bccc-56e196e6156b admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:08:25.371489 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 900/1832] 10.222.0.46 () {60 vars in 1483 bytes} [Mon Oct 10 12:08:25 2022] PUT /identity/v3/projects/45f32ff86e37403abcda138fa40c8374/users/0886c8e7e1814e51a5b9b20807d33098/roles/0d822a0674b6495fac623877a3b38025 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:08:26.935987 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7de7c02-9bbd-4973-aba5-bea6defd7f58 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:08:26.936574 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7de7c02-9bbd-4973-aba5-bea6defd7f58 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:08:26.936574 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7de7c02-9bbd-4973-aba5-bea6defd7f58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:08:26.975427 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:08:27.004112 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-f7de7c02-9bbd-4973-aba5-bea6defd7f58 None None] MFA Rules not processed for user `0886c8e7e1814e51a5b9b20807d33098`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:08:27.005526 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-f7de7c02-9bbd-4973-aba5-bea6defd7f58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:08:27.116887 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 933/1833] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:08:26 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:08:27.127632 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3d58c6-3010-487f-bccc-56e196e6156b admin admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:08:27.131531 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-db3d58c6-3010-487f-bccc-56e196e6156b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:08:27.235568 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3d58c6-3010-487f-bccc-56e196e6156b admin admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:08:27.239877 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ad507e-6999-4660-b23c-7c163303612d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3ae093470484b93b23dcfae1657df9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad3772f7b6a47c28b7ba37a9316a3c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:08:27.241318 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ad507e-6999-4660-b23c-7c163303612d service cinder] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:08:27.241775 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ad507e-6999-4660-b23c-7c163303612d service cinder] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:08:27.242141 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ad507e-6999-4660-b23c-7c163303612d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:08:27.329498 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ad507e-6999-4660-b23c-7c163303612d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:08:27.331140 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ad507e-6999-4660-b23c-7c163303612d service cinder] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:08:27.342285 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 901/1834] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 10 12:08:27 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 216 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 10 12:08:31.217533 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7de7c02-9bbd-4973-aba5-bea6defd7f58 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:08:31.235488 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7de7c02-9bbd-4973-aba5-bea6defd7f58 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:08:31.243826 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf74e55-4f3a-4b52-8757-834565cc4288 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3ae093470484b93b23dcfae1657df9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad3772f7b6a47c28b7ba37a9316a3c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:08:31.245317 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf74e55-4f3a-4b52-8757-834565cc4288 service cinder] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:08:31.245997 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf74e55-4f3a-4b52-8757-834565cc4288 service cinder] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:08:31.246893 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf74e55-4f3a-4b52-8757-834565cc4288 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:08:31.263171 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf74e55-4f3a-4b52-8757-834565cc4288 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:08:31.264682 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf74e55-4f3a-4b52-8757-834565cc4288 service cinder] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:08:31.284818 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 934/1835] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 10 12:08:31 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:08:34.486178 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ad507e-6999-4660-b23c-7c163303612d service cinder] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:08:34.597139 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ad507e-6999-4660-b23c-7c163303612d service cinder] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:08:34.603188 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c40eec-0adb-47fa-b9aa-32c2d5fdfe9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:08:34.603754 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c40eec-0adb-47fa-b9aa-32c2d5fdfe9d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:08:34.604036 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c40eec-0adb-47fa-b9aa-32c2d5fdfe9d admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:08:34.604307 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c40eec-0adb-47fa-b9aa-32c2d5fdfe9d admin admin] PATH_INFO: `/v3/users/47ca6459e866442e94df1cb09e6cc5a0` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:08:34.605729 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c40eec-0adb-47fa-b9aa-32c2d5fdfe9d admin admin] RBAC: Authorizing `identity:delete_user(user_id=47ca6459e866442e94df1cb09e6cc5a0)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:08:34.607720 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c40eec-0adb-47fa-b9aa-32c2d5fdfe9d admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:08:34.899986 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=93,keystone:INSERT=2,keystone:DELETE=5 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:08:34.919609 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-50c40eec-0adb-47fa-b9aa-32c2d5fdfe9d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '47ca6459e866442e94df1cb09e6cc5a0'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:08:34.932703 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-50c40eec-0adb-47fa-b9aa-32c2d5fdfe9d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '47ca6459e866442e94df1cb09e6cc5a0'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:08:34.935862 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-50c40eec-0adb-47fa-b9aa-32c2d5fdfe9d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '47ca6459e866442e94df1cb09e6cc5a0'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:08:35.028980 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-50c40eec-0adb-47fa-b9aa-32c2d5fdfe9d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '47ca6459e866442e94df1cb09e6cc5a0'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:08:35.048108 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-50c40eec-0adb-47fa-b9aa-32c2d5fdfe9d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '47ca6459e866442e94df1cb09e6cc5a0'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:08:35.059876 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 902/1836] 10.222.0.46 () {58 vars in 1224 bytes} [Mon Oct 10 12:08:34 2022] DELETE /identity/v3/users/47ca6459e866442e94df1cb09e6cc5a0 => generated 0 bytes in 576 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:08:35.066572 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfadac6-3297-4527-8962-d14190b78825 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:08:35.067558 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfadac6-3297-4527-8962-d14190b78825 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:08:35.067989 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfadac6-3297-4527-8962-d14190b78825 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:08:35.101153 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-8dfadac6-3297-4527-8962-d14190b78825 None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:08:35.102625 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-8dfadac6-3297-4527-8962-d14190b78825 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:08:35.237454 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 935/1837] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:08:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:08:35.252348 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c40eec-0adb-47fa-b9aa-32c2d5fdfe9d admin admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:08:35.254767 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-50c40eec-0adb-47fa-b9aa-32c2d5fdfe9d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:08:35.355881 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c40eec-0adb-47fa-b9aa-32c2d5fdfe9d admin admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:08:35.361301 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467d8dbc-b463-411b-bed1-f3bfee5a651c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58b1c7c85302421cba823ace4201861f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad3772f7b6a47c28b7ba37a9316a3c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:08:35.361943 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467d8dbc-b463-411b-bed1-f3bfee5a651c service neutron] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:08:35.362347 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467d8dbc-b463-411b-bed1-f3bfee5a651c service neutron] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:08:35.362794 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467d8dbc-b463-411b-bed1-f3bfee5a651c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:08:35.376672 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467d8dbc-b463-411b-bed1-f3bfee5a651c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:08:35.379413 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467d8dbc-b463-411b-bed1-f3bfee5a651c service neutron] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:08:35.411465 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 903/1838] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 10 12:08:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 10 12:08:36.079530 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfadac6-3297-4527-8962-d14190b78825 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:08:36.081647 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-8dfadac6-3297-4527-8962-d14190b78825 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:08:36.105411 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfadac6-3297-4527-8962-d14190b78825 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:08:36.115267 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3a8afb-9e60-4239-b4e4-91cb429dd468 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:08:36.117691 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3a8afb-9e60-4239-b4e4-91cb429dd468 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:08:36.118456 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3a8afb-9e60-4239-b4e4-91cb429dd468 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:08:36.119237 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3a8afb-9e60-4239-b4e4-91cb429dd468 admin admin] PATH_INFO: `/v3/projects/ec6ecce071ae4b72a1ee50e780f65112` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:08:36.121756 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3a8afb-9e60-4239-b4e4-91cb429dd468 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ec6ecce071ae4b72a1ee50e780f65112)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:08:36.125516 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3a8afb-9e60-4239-b4e4-91cb429dd468 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:08:36.310030 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-cd3a8afb-9e60-4239-b4e4-91cb429dd468 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ec6ecce071ae4b72a1ee50e780f65112', 'request_id': 'req-cd3a8afb-9e60-4239-b4e4-91cb429dd468'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:08:36.316474 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-cd3a8afb-9e60-4239-b4e4-91cb429dd468 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ec6ecce071ae4b72a1ee50e780f65112', 'request_id': 'req-cd3a8afb-9e60-4239-b4e4-91cb429dd468'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:08:36.323877 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-cd3a8afb-9e60-4239-b4e4-91cb429dd468 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ec6ecce071ae4b72a1ee50e780f65112', 'request_id': 'req-cd3a8afb-9e60-4239-b4e4-91cb429dd468'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:08:36.406591 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467d8dbc-b463-411b-bed1-f3bfee5a651c service neutron] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:08:36.408370 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-467d8dbc-b463-411b-bed1-f3bfee5a651c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:08:36.424217 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467d8dbc-b463-411b-bed1-f3bfee5a651c service neutron] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:08:36.479937 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-cd3a8afb-9e60-4239-b4e4-91cb429dd468 admin admin] The token cache is being invalidate because project ec6ecce071ae4b72a1ee50e780f65112 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81677) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 10 12:08:36.480762 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-cd3a8afb-9e60-4239-b4e4-91cb429dd468 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:08:36.481913 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-cd3a8afb-9e60-4239-b4e4-91cb429dd468 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:08:36.485169 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 936/1839] 10.222.0.46 () {58 vars in 1233 bytes} [Mon Oct 10 12:08:36 2022] DELETE /identity/v3/projects/ec6ecce071ae4b72a1ee50e780f65112 => generated 0 bytes in 408 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:08:36.490172 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3a8afb-9e60-4239-b4e4-91cb429dd468 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:08:36.492711 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-cd3a8afb-9e60-4239-b4e4-91cb429dd468 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:08:36.499903 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:08:36.508118 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7207a07-ca0d-41d6-8993-e272cb481692 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:08:36.509217 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7207a07-ca0d-41d6-8993-e272cb481692 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:08:36.509640 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7207a07-ca0d-41d6-8993-e272cb481692 admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:08:36.510008 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7207a07-ca0d-41d6-8993-e272cb481692 admin admin] PATH_INFO: `/v3/users/dbfe4e28b2fb46428a20bb0c2fb15c1f` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:08:36.511974 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7207a07-ca0d-41d6-8993-e272cb481692 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dbfe4e28b2fb46428a20bb0c2fb15c1f)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:08:36.514545 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7207a07-ca0d-41d6-8993-e272cb481692 admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:08:36.603969 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3a8afb-9e60-4239-b4e4-91cb429dd468 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:08:36.613246 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363a4caa-b90c-4ef1-ac2d-993202e84de1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:08:36.614392 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363a4caa-b90c-4ef1-ac2d-993202e84de1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:08:36.614843 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363a4caa-b90c-4ef1-ac2d-993202e84de1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:08:36.615506 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363a4caa-b90c-4ef1-ac2d-993202e84de1 admin admin] PATH_INFO: `/v3/users/5f8ca4578e29466586f011aeafefc1e2` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:08:36.618385 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363a4caa-b90c-4ef1-ac2d-993202e84de1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5f8ca4578e29466586f011aeafefc1e2)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:08:36.622888 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363a4caa-b90c-4ef1-ac2d-993202e84de1 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:08:36.978741 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-d7207a07-ca0d-41d6-8993-e272cb481692 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dbfe4e28b2fb46428a20bb0c2fb15c1f'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:08:36.991767 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:08:36.998442 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-d7207a07-ca0d-41d6-8993-e272cb481692 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dbfe4e28b2fb46428a20bb0c2fb15c1f'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:08:37.001675 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-d7207a07-ca0d-41d6-8993-e272cb481692 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dbfe4e28b2fb46428a20bb0c2fb15c1f'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:08:37.016391 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-363a4caa-b90c-4ef1-ac2d-993202e84de1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5f8ca4578e29466586f011aeafefc1e2'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:08:37.038244 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-363a4caa-b90c-4ef1-ac2d-993202e84de1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5f8ca4578e29466586f011aeafefc1e2'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:08:37.041340 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-363a4caa-b90c-4ef1-ac2d-993202e84de1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5f8ca4578e29466586f011aeafefc1e2'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:08:37.085821 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-d7207a07-ca0d-41d6-8993-e272cb481692 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dbfe4e28b2fb46428a20bb0c2fb15c1f'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:08:37.115171 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-d7207a07-ca0d-41d6-8993-e272cb481692 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dbfe4e28b2fb46428a20bb0c2fb15c1f'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:08:37.134657 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 904/1840] 10.222.0.46 () {58 vars in 1224 bytes} [Mon Oct 10 12:08:36 2022] DELETE /identity/v3/users/dbfe4e28b2fb46428a20bb0c2fb15c1f => generated 0 bytes in 724 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 10 12:08:37.137382 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fdedd6-dddf-4855-a5da-d3794a20e941 None None] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:08:37.137845 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fdedd6-dddf-4855-a5da-d3794a20e941 None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:08:37.138319 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fdedd6-dddf-4855-a5da-d3794a20e941 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:08:37.184325 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-363a4caa-b90c-4ef1-ac2d-993202e84de1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5f8ca4578e29466586f011aeafefc1e2'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:08:37.188835 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.auth.core [None req-88fdedd6-dddf-4855-a5da-d3794a20e941 None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:08:37.191648 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-88fdedd6-dddf-4855-a5da-d3794a20e941 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:08:37.259929 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-363a4caa-b90c-4ef1-ac2d-993202e84de1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5f8ca4578e29466586f011aeafefc1e2'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:08:37.297969 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 937/1841] 10.222.0.46 () {58 vars in 1224 bytes} [Mon Oct 10 12:08:36 2022] DELETE /identity/v3/users/5f8ca4578e29466586f011aeafefc1e2 => generated 0 bytes in 808 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 10 12:08:37.393307 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 905/1842] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:08:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 10 12:08:37.530862 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363a4caa-b90c-4ef1-ac2d-993202e84de1 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:08:37.534789 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-363a4caa-b90c-4ef1-ac2d-993202e84de1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:08:37.683577 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363a4caa-b90c-4ef1-ac2d-993202e84de1 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:08:37.690134 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf5c717-2c72-4bdd-86f2-33ad242d763f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58b1c7c85302421cba823ace4201861f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad3772f7b6a47c28b7ba37a9316a3c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:08:37.691904 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf5c717-2c72-4bdd-86f2-33ad242d763f service neutron] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:08:37.692603 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf5c717-2c72-4bdd-86f2-33ad242d763f service neutron] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:08:37.693226 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf5c717-2c72-4bdd-86f2-33ad242d763f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:08:37.714956 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb11edd8-a38e-4130-b9b8-a0834c5c9323 None None] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:08:37.716070 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb11edd8-a38e-4130-b9b8-a0834c5c9323 None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:08:37.717393 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb11edd8-a38e-4130-b9b8-a0834c5c9323 None None] PATH_INFO: `/` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:08:37.721410 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 906/1843] 10.222.0.46 () {54 vars in 921 bytes} [Mon Oct 10 12:08:37 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 12:08:37.761850 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74703ef-9944-474c-a95f-8279649b8aa2 None None] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:08:37.762485 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74703ef-9944-474c-a95f-8279649b8aa2 None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:08:37.763133 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74703ef-9944-474c-a95f-8279649b8aa2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:08:37.805390 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:08:37.825867 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.auth.core [None req-f74703ef-9944-474c-a95f-8279649b8aa2 None None] MFA Rules not processed for user `5f130bbb69e546ce99b4e14de81d7e54`. Rule list: `[]` (Enabled: `True`). {{(pid=81676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:08:37.830455 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-f74703ef-9944-474c-a95f-8279649b8aa2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:08:37.888645 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf5c717-2c72-4bdd-86f2-33ad242d763f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:08:37.893500 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf5c717-2c72-4bdd-86f2-33ad242d763f service neutron] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:08:37.926541 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 938/1844] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 10 12:08:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:08:38.033726 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 907/1845] 10.222.0.46 () {58 vars in 1018 bytes} [Mon Oct 10 12:08:37 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 274 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 10 12:08:38.564257 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf5c717-2c72-4bdd-86f2-33ad242d763f service neutron] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:08:38.567531 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-ddf5c717-2c72-4bdd-86f2-33ad242d763f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:08:38.598938 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf5c717-2c72-4bdd-86f2-33ad242d763f service neutron] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:08:38.612182 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5acb9c-7ced-4231-8326-00f287fb34b8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f130bbb69e546ce99b4e14de81d7e54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad3772f7b6a47c28b7ba37a9316a3c1', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:08:38.613254 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5acb9c-7ced-4231-8326-00f287fb34b8 service nova] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:08:38.614051 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5acb9c-7ced-4231-8326-00f287fb34b8 service nova] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:08:38.614600 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5acb9c-7ced-4231-8326-00f287fb34b8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:08:38.632560 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5acb9c-7ced-4231-8326-00f287fb34b8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:08:38.635784 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5acb9c-7ced-4231-8326-00f287fb34b8 service nova] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:08:38.674329 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 939/1846] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 10 12:08:38 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:08:38.962499 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74703ef-9944-474c-a95f-8279649b8aa2 None None] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:08:38.964722 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-f74703ef-9944-474c-a95f-8279649b8aa2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:08:38.995460 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74703ef-9944-474c-a95f-8279649b8aa2 None None] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:08:39.004950 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63dc7cb8-7f2c-493c-ad6c-f3b46b04bdfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:08:39.006036 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63dc7cb8-7f2c-493c-ad6c-f3b46b04bdfc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:08:39.006601 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63dc7cb8-7f2c-493c-ad6c-f3b46b04bdfc admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:08:39.007211 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63dc7cb8-7f2c-493c-ad6c-f3b46b04bdfc admin admin] PATH_INFO: `/v3/projects/04b64d7d543a4c24892f9d1747a94735` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:08:39.009637 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63dc7cb8-7f2c-493c-ad6c-f3b46b04bdfc admin admin] RBAC: Authorizing `identity:delete_project(project_id=04b64d7d543a4c24892f9d1747a94735)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:08:39.011291 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5acb9c-7ced-4231-8326-00f287fb34b8 service nova] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:08:39.013536 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-1e5acb9c-7ced-4231-8326-00f287fb34b8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:08:39.014456 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63dc7cb8-7f2c-493c-ad6c-f3b46b04bdfc admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:08:39.034825 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5acb9c-7ced-4231-8326-00f287fb34b8 service nova] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:08:39.046341 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065410a1-8a63-4ffe-910e-f74d0202049b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:08:39.047495 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065410a1-8a63-4ffe-910e-f74d0202049b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:08:39.047985 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065410a1-8a63-4ffe-910e-f74d0202049b admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:08:39.048564 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065410a1-8a63-4ffe-910e-f74d0202049b admin admin] PATH_INFO: `/v3/projects/0a990bbe822443c9b5e465bb6a97e7f8` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:08:39.050986 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065410a1-8a63-4ffe-910e-f74d0202049b admin admin] RBAC: Authorizing `identity:delete_project(project_id=0a990bbe822443c9b5e465bb6a97e7f8)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:08:39.056135 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065410a1-8a63-4ffe-910e-f74d0202049b admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:08:39.206192 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-63dc7cb8-7f2c-493c-ad6c-f3b46b04bdfc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '04b64d7d543a4c24892f9d1747a94735', 'request_id': 'req-63dc7cb8-7f2c-493c-ad6c-f3b46b04bdfc'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:08:39.221254 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-63dc7cb8-7f2c-493c-ad6c-f3b46b04bdfc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '04b64d7d543a4c24892f9d1747a94735', 'request_id': 'req-63dc7cb8-7f2c-493c-ad6c-f3b46b04bdfc'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:08:39.223765 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=90,keystone:INSERT=1,keystone:DELETE=9 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:08:39.241085 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-63dc7cb8-7f2c-493c-ad6c-f3b46b04bdfc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '04b64d7d543a4c24892f9d1747a94735', 'request_id': 'req-63dc7cb8-7f2c-493c-ad6c-f3b46b04bdfc'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:08:39.295863 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-065410a1-8a63-4ffe-910e-f74d0202049b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0a990bbe822443c9b5e465bb6a97e7f8', 'request_id': 'req-065410a1-8a63-4ffe-910e-f74d0202049b'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:08:39.308141 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-065410a1-8a63-4ffe-910e-f74d0202049b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0a990bbe822443c9b5e465bb6a97e7f8', 'request_id': 'req-065410a1-8a63-4ffe-910e-f74d0202049b'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:08:39.325808 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-065410a1-8a63-4ffe-910e-f74d0202049b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0a990bbe822443c9b5e465bb6a97e7f8', 'request_id': 'req-065410a1-8a63-4ffe-910e-f74d0202049b'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:08:39.368902 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-63dc7cb8-7f2c-493c-ad6c-f3b46b04bdfc admin admin] The token cache is being invalidate because project 04b64d7d543a4c24892f9d1747a94735 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81676) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 10 12:08:39.370426 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-63dc7cb8-7f2c-493c-ad6c-f3b46b04bdfc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:08:39.371553 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-63dc7cb8-7f2c-493c-ad6c-f3b46b04bdfc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:08:39.374120 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 908/1847] 10.222.0.46 () {58 vars in 1233 bytes} [Mon Oct 10 12:08:38 2022] DELETE /identity/v3/projects/04b64d7d543a4c24892f9d1747a94735 => generated 0 bytes in 412 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:08:39.382065 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c761d4c4-7ef9-4a75-9db5-51c89a9f7496 None None] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:08:39.382502 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c761d4c4-7ef9-4a75-9db5-51c89a9f7496 None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:08:39.382947 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c761d4c4-7ef9-4a75-9db5-51c89a9f7496 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:08:39.409928 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.auth.core [None req-c761d4c4-7ef9-4a75-9db5-51c89a9f7496 None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:08:39.411226 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-c761d4c4-7ef9-4a75-9db5-51c89a9f7496 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:08:39.491308 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-065410a1-8a63-4ffe-910e-f74d0202049b admin admin] The token cache is being invalidate because project 0a990bbe822443c9b5e465bb6a97e7f8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81677) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 10 12:08:39.492616 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-065410a1-8a63-4ffe-910e-f74d0202049b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:08:39.499424 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-065410a1-8a63-4ffe-910e-f74d0202049b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:08:39.504975 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 940/1848] 10.222.0.46 () {58 vars in 1233 bytes} [Mon Oct 10 12:08:39 2022] DELETE /identity/v3/projects/0a990bbe822443c9b5e465bb6a97e7f8 => generated 0 bytes in 497 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:08:39.512181 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065410a1-8a63-4ffe-910e-f74d0202049b admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:08:39.517482 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-065410a1-8a63-4ffe-910e-f74d0202049b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:08:39.543221 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 909/1849] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:08:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:08:39.559498 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6361edaf-86ea-4375-a5a2-08be24ccfd24 None None] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:08:39.560536 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6361edaf-86ea-4375-a5a2-08be24ccfd24 None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:08:39.562792 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6361edaf-86ea-4375-a5a2-08be24ccfd24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:08:39.628062 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.auth.core [None req-6361edaf-86ea-4375-a5a2-08be24ccfd24 None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:08:39.630099 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-6361edaf-86ea-4375-a5a2-08be24ccfd24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:08:39.665070 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 910/1850] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:08:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 10 12:08:39.667352 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6361edaf-86ea-4375-a5a2-08be24ccfd24 None None] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:08:39.694253 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6361edaf-86ea-4375-a5a2-08be24ccfd24 None None] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:08:39.708313 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9075ad-91c6-4212-8745-20027714ee39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:08:39.709891 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9075ad-91c6-4212-8745-20027714ee39 admin admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:08:39.710589 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9075ad-91c6-4212-8745-20027714ee39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:08:39.711118 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9075ad-91c6-4212-8745-20027714ee39 admin admin] PATH_INFO: `/v3/domains` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:08:39.712708 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9075ad-91c6-4212-8745-20027714ee39 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 12:08:39.713275 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9075ad-91c6-4212-8745-20027714ee39 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:08:39.719608 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9075ad-91c6-4212-8745-20027714ee39 admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:08:39.725876 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065410a1-8a63-4ffe-910e-f74d0202049b admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:08:39.732239 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aeb6742-fb77-4bf6-b6d8-79a4b74dcf95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:08:39.733279 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aeb6742-fb77-4bf6-b6d8-79a4b74dcf95 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:08:39.733783 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aeb6742-fb77-4bf6-b6d8-79a4b74dcf95 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:08:39.734251 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aeb6742-fb77-4bf6-b6d8-79a4b74dcf95 admin admin] PATH_INFO: `/v3/users/7f1d255fdc844eb5893a79309d8601f5` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:08:39.735910 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aeb6742-fb77-4bf6-b6d8-79a4b74dcf95 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7f1d255fdc844eb5893a79309d8601f5)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:08:39.738198 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aeb6742-fb77-4bf6-b6d8-79a4b74dcf95 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:08:39.751720 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 911/1851] 10.222.0.46 () {58 vars in 1153 bytes} [Mon Oct 10 12:08:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 12:08:39.760081 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9075ad-91c6-4212-8745-20027714ee39 admin admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:08:39.790793 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9075ad-91c6-4212-8745-20027714ee39 admin admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:08:39.804739 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791ed3e5-8409-41cf-b8dd-6acda5d30350 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:08:39.810305 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791ed3e5-8409-41cf-b8dd-6acda5d30350 admin admin] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:08:39.810305 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791ed3e5-8409-41cf-b8dd-6acda5d30350 admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:08:39.810305 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791ed3e5-8409-41cf-b8dd-6acda5d30350 admin admin] PATH_INFO: `/v3/projects` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:08:39.810305 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791ed3e5-8409-41cf-b8dd-6acda5d30350 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:08:39.812905 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791ed3e5-8409-41cf-b8dd-6acda5d30350 admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:08:39.956462 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 912/1852] 10.222.0.46 () {60 vars in 1153 bytes} [Mon Oct 10 12:08:39 2022] POST /identity/v3/projects => generated 377 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 12:08:39.958734 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791ed3e5-8409-41cf-b8dd-6acda5d30350 admin admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:08:40.057834 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:08:40.068062 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-7aeb6742-fb77-4bf6-b6d8-79a4b74dcf95 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7f1d255fdc844eb5893a79309d8601f5'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:08:40.115549 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-7aeb6742-fb77-4bf6-b6d8-79a4b74dcf95 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7f1d255fdc844eb5893a79309d8601f5'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:08:40.118221 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791ed3e5-8409-41cf-b8dd-6acda5d30350 admin admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:08:40.122166 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-7aeb6742-fb77-4bf6-b6d8-79a4b74dcf95 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7f1d255fdc844eb5893a79309d8601f5'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:08:40.149781 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6eee6bd-3a33-4615-b4a2-47a6bc8fca55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:08:40.151013 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6eee6bd-3a33-4615-b4a2-47a6bc8fca55 admin admin] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:08:40.152963 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6eee6bd-3a33-4615-b4a2-47a6bc8fca55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:08:40.158533 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6eee6bd-3a33-4615-b4a2-47a6bc8fca55 admin admin] PATH_INFO: `/v3/users` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:08:40.160449 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6eee6bd-3a33-4615-b4a2-47a6bc8fca55 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:08:40.166458 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6eee6bd-3a33-4615-b4a2-47a6bc8fca55 admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:08:40.221687 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-7aeb6742-fb77-4bf6-b6d8-79a4b74dcf95 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7f1d255fdc844eb5893a79309d8601f5'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:08:40.270028 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-7aeb6742-fb77-4bf6-b6d8-79a4b74dcf95 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7f1d255fdc844eb5893a79309d8601f5'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:08:40.288529 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 941/1853] 10.222.0.46 () {58 vars in 1224 bytes} [Mon Oct 10 12:08:39 2022] DELETE /identity/v3/users/7f1d255fdc844eb5893a79309d8601f5 => generated 0 bytes in 777 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 10 12:08:40.309368 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 913/1854] 10.222.0.46 () {60 vars in 1144 bytes} [Mon Oct 10 12:08:39 2022] POST /identity/v3/users => generated 289 bytes in 352 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 12:08:40.321466 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aeb6742-fb77-4bf6-b6d8-79a4b74dcf95 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:08:40.324098 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-7aeb6742-fb77-4bf6-b6d8-79a4b74dcf95 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:08:40.454845 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aeb6742-fb77-4bf6-b6d8-79a4b74dcf95 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:08:40.465377 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba96836-4d9f-4a81-8340-a5fc997ba74f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:08:40.466527 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba96836-4d9f-4a81-8340-a5fc997ba74f admin admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:08:40.467012 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba96836-4d9f-4a81-8340-a5fc997ba74f admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:08:40.468309 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba96836-4d9f-4a81-8340-a5fc997ba74f admin admin] PATH_INFO: `/v3/roles` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:08:40.468793 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba96836-4d9f-4a81-8340-a5fc997ba74f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:08:40.472060 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba96836-4d9f-4a81-8340-a5fc997ba74f admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:08:40.509004 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 942/1855] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 10 12:08:40 2022] GET /identity/v3/roles => generated 1420 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 12:08:40.529418 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6eee6bd-3a33-4615-b4a2-47a6bc8fca55 admin admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:08:40.556293 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6eee6bd-3a33-4615-b4a2-47a6bc8fca55 admin admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:08:40.565672 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af67991e-f60a-443a-8174-fa719c5f46d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:08:40.566820 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af67991e-f60a-443a-8174-fa719c5f46d4 admin admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:08:40.567321 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af67991e-f60a-443a-8174-fa719c5f46d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:08:40.567814 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af67991e-f60a-443a-8174-fa719c5f46d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:08:40.569576 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af67991e-f60a-443a-8174-fa719c5f46d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:08:40.572653 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af67991e-f60a-443a-8174-fa719c5f46d4 admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:08:40.600979 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 914/1856] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 10 12:08:40 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 12:08:40.607514 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba96836-4d9f-4a81-8340-a5fc997ba74f admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:08:40.632093 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba96836-4d9f-4a81-8340-a5fc997ba74f admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:08:40.644473 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf811779-608d-43f1-aa8d-59d38dfc892f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:08:40.648005 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf811779-608d-43f1-aa8d-59d38dfc892f admin admin] REQUEST_METHOD: `PUT` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:08:40.648827 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf811779-608d-43f1-aa8d-59d38dfc892f admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:08:40.649508 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf811779-608d-43f1-aa8d-59d38dfc892f admin admin] PATH_INFO: `/v3/projects/d80eb37d6d7a4475bcdb1594514faba1/users/2c1cd9c6ca5b4783bbf797ae125a9e79/roles/b2cb2a6e818e47338ce071e6fedfc673` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:08:40.681052 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf811779-608d-43f1-aa8d-59d38dfc892f admin admin] RBAC: Authorizing `identity:create_grant(project_id=d80eb37d6d7a4475bcdb1594514faba1, user_id=2c1cd9c6ca5b4783bbf797ae125a9e79, role_id=b2cb2a6e818e47338ce071e6fedfc673)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:08:40.687591 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf811779-608d-43f1-aa8d-59d38dfc892f admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:08:40.821385 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 943/1857] 10.222.0.46 () {60 vars in 1483 bytes} [Mon Oct 10 12:08:40 2022] PUT /identity/v3/projects/d80eb37d6d7a4475bcdb1594514faba1/users/2c1cd9c6ca5b4783bbf797ae125a9e79/roles/b2cb2a6e818e47338ce071e6fedfc673 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:08:40.847944 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8dd1461-3203-4407-b144-5ca26f05c6eb None None] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:08:40.848293 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8dd1461-3203-4407-b144-5ca26f05c6eb None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:08:40.848583 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8dd1461-3203-4407-b144-5ca26f05c6eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:08:40.906090 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.auth.core [None req-f8dd1461-3203-4407-b144-5ca26f05c6eb None None] MFA Rules not processed for user `2c1cd9c6ca5b4783bbf797ae125a9e79`. Rule list: `[]` (Enabled: `True`). {{(pid=81676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:08:40.909260 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-f8dd1461-3203-4407-b144-5ca26f05c6eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:08:40.936374 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=94,keystone:INSERT=5,keystone:DELETE=1 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:08:41.073911 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 915/1858] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:08:40 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:08:41.096662 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf811779-608d-43f1-aa8d-59d38dfc892f admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:08:41.099144 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-cf811779-608d-43f1-aa8d-59d38dfc892f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:08:41.270041 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf811779-608d-43f1-aa8d-59d38dfc892f admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:08:41.274516 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ae0800-27bf-4870-98d2-d2f4dc00b4ce service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '124f4aa8ab244ca2aadf4f2614df51e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad3772f7b6a47c28b7ba37a9316a3c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:08:41.275190 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ae0800-27bf-4870-98d2-d2f4dc00b4ce service glance] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:08:41.275479 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ae0800-27bf-4870-98d2-d2f4dc00b4ce service glance] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:08:41.275740 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ae0800-27bf-4870-98d2-d2f4dc00b4ce service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:08:41.288286 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ae0800-27bf-4870-98d2-d2f4dc00b4ce service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:08:41.289900 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ae0800-27bf-4870-98d2-d2f4dc00b4ce service glance] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:08:41.301080 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 944/1859] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 10 12:08:41 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:08:41.689988 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8dd1461-3203-4407-b144-5ca26f05c6eb None None] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:08:41.691813 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-f8dd1461-3203-4407-b144-5ca26f05c6eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:08:41.921549 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8dd1461-3203-4407-b144-5ca26f05c6eb None None] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:08:41.941021 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea79a6d6-93d8-433a-accd-6f4edb841d91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:08:41.942158 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea79a6d6-93d8-433a-accd-6f4edb841d91 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:08:41.942693 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea79a6d6-93d8-433a-accd-6f4edb841d91 admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:08:41.943752 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea79a6d6-93d8-433a-accd-6f4edb841d91 admin admin] PATH_INFO: `/v3/projects/4cb37ff0b2cd4107a5a24f81d7e90fb5` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:08:41.950515 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea79a6d6-93d8-433a-accd-6f4edb841d91 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4cb37ff0b2cd4107a5a24f81d7e90fb5)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:08:41.954455 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea79a6d6-93d8-433a-accd-6f4edb841d91 admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:08:42.154767 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-ea79a6d6-93d8-433a-accd-6f4edb841d91 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4cb37ff0b2cd4107a5a24f81d7e90fb5', 'request_id': 'req-ea79a6d6-93d8-433a-accd-6f4edb841d91'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:08:42.186318 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-ea79a6d6-93d8-433a-accd-6f4edb841d91 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4cb37ff0b2cd4107a5a24f81d7e90fb5', 'request_id': 'req-ea79a6d6-93d8-433a-accd-6f4edb841d91'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:08:42.201743 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-ea79a6d6-93d8-433a-accd-6f4edb841d91 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4cb37ff0b2cd4107a5a24f81d7e90fb5', 'request_id': 'req-ea79a6d6-93d8-433a-accd-6f4edb841d91'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:08:42.333918 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-ea79a6d6-93d8-433a-accd-6f4edb841d91 admin admin] The token cache is being invalidate because project 4cb37ff0b2cd4107a5a24f81d7e90fb5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81676) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 10 12:08:42.334753 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-ea79a6d6-93d8-433a-accd-6f4edb841d91 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:08:42.343741 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-ea79a6d6-93d8-433a-accd-6f4edb841d91 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:08:42.348047 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 916/1860] 10.222.0.46 () {58 vars in 1233 bytes} [Mon Oct 10 12:08:41 2022] DELETE /identity/v3/projects/4cb37ff0b2cd4107a5a24f81d7e90fb5 => generated 0 bytes in 659 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:08:42.357098 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246b1944-3def-4347-a1bb-709952a6e16f None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:08:42.357861 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246b1944-3def-4347-a1bb-709952a6e16f None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:08:42.358216 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246b1944-3def-4347-a1bb-709952a6e16f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:08:42.405438 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-246b1944-3def-4347-a1bb-709952a6e16f None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:08:42.408406 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-246b1944-3def-4347-a1bb-709952a6e16f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:08:42.426123 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:DELETE=4,keystone:SELECT=94,keystone:INSERT=2 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:08:42.557375 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 945/1861] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:08:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:08:42.567262 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20325ec1-4627-4fe2-98d0-656c4294b721 None None] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:08:42.567542 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20325ec1-4627-4fe2-98d0-656c4294b721 None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:08:42.567841 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20325ec1-4627-4fe2-98d0-656c4294b721 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:08:42.593788 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.auth.core [None req-20325ec1-4627-4fe2-98d0-656c4294b721 None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:08:42.596760 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-20325ec1-4627-4fe2-98d0-656c4294b721 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:08:42.627159 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 917/1862] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:08:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 10 12:08:42.629251 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246b1944-3def-4347-a1bb-709952a6e16f None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:08:42.648277 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246b1944-3def-4347-a1bb-709952a6e16f None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:08:42.660938 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6ee0ae-4d89-49f8-b2f3-b35e6155febf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:08:42.662634 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6ee0ae-4d89-49f8-b2f3-b35e6155febf admin admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:08:42.663318 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6ee0ae-4d89-49f8-b2f3-b35e6155febf admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:08:42.663888 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6ee0ae-4d89-49f8-b2f3-b35e6155febf admin admin] PATH_INFO: `/v3/domains` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:08:42.666638 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6ee0ae-4d89-49f8-b2f3-b35e6155febf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 12:08:42.667471 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6ee0ae-4d89-49f8-b2f3-b35e6155febf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:08:42.670569 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6ee0ae-4d89-49f8-b2f3-b35e6155febf admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:08:42.695308 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 946/1863] 10.222.0.46 () {58 vars in 1153 bytes} [Mon Oct 10 12:08:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 12:08:42.700760 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20325ec1-4627-4fe2-98d0-656c4294b721 None None] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:08:42.721365 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20325ec1-4627-4fe2-98d0-656c4294b721 None None] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:08:42.731746 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b089215-0fb7-4015-b904-13e06dca4c71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:08:42.733276 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b089215-0fb7-4015-b904-13e06dca4c71 admin admin] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:08:42.733276 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b089215-0fb7-4015-b904-13e06dca4c71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:08:42.736032 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b089215-0fb7-4015-b904-13e06dca4c71 admin admin] PATH_INFO: `/v3/projects` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:08:42.736032 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b089215-0fb7-4015-b904-13e06dca4c71 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:08:42.741034 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b089215-0fb7-4015-b904-13e06dca4c71 admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:08:42.960044 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 918/1864] 10.222.0.46 () {60 vars in 1153 bytes} [Mon Oct 10 12:08:42 2022] POST /identity/v3/projects => generated 379 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 12:08:42.962529 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6ee0ae-4d89-49f8-b2f3-b35e6155febf admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:08:42.983648 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6ee0ae-4d89-49f8-b2f3-b35e6155febf admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:08:43.111784 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78b11ef-da18-4395-990d-116d20791e4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:08:43.112620 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78b11ef-da18-4395-990d-116d20791e4d admin admin] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:08:43.113114 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78b11ef-da18-4395-990d-116d20791e4d admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:08:43.113593 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78b11ef-da18-4395-990d-116d20791e4d admin admin] PATH_INFO: `/v3/users` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:08:43.114515 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78b11ef-da18-4395-990d-116d20791e4d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:08:43.117798 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78b11ef-da18-4395-990d-116d20791e4d admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:08:43.260162 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 947/1865] 10.222.0.46 () {60 vars in 1144 bytes} [Mon Oct 10 12:08:42 2022] POST /identity/v3/users => generated 290 bytes in 298 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 12:08:43.265736 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b089215-0fb7-4015-b904-13e06dca4c71 admin admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:08:43.293738 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b089215-0fb7-4015-b904-13e06dca4c71 admin admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:08:43.299945 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4eba70-b5b9-4932-bc2f-845d537cb101 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:08:43.300735 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4eba70-b5b9-4932-bc2f-845d537cb101 admin admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:08:43.301685 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4eba70-b5b9-4932-bc2f-845d537cb101 admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:08:43.302752 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4eba70-b5b9-4932-bc2f-845d537cb101 admin admin] PATH_INFO: `/v3/roles` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:08:43.303597 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4eba70-b5b9-4932-bc2f-845d537cb101 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:08:43.332976 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4eba70-b5b9-4932-bc2f-845d537cb101 admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:08:43.366973 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 919/1866] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 10 12:08:43 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 12:08:43.371782 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78b11ef-da18-4395-990d-116d20791e4d admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:08:43.391392 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78b11ef-da18-4395-990d-116d20791e4d admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:08:43.402326 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2310c8-cf11-4c44-979b-04c853e94f39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:08:43.403919 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2310c8-cf11-4c44-979b-04c853e94f39 admin admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:08:43.404660 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2310c8-cf11-4c44-979b-04c853e94f39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:08:43.405359 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2310c8-cf11-4c44-979b-04c853e94f39 admin admin] PATH_INFO: `/v3/roles` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:08:43.406592 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2310c8-cf11-4c44-979b-04c853e94f39 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:08:43.410426 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2310c8-cf11-4c44-979b-04c853e94f39 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:08:43.433576 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 948/1867] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 10 12:08:43 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 12:08:43.444740 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4eba70-b5b9-4932-bc2f-845d537cb101 admin admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:08:43.470282 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:08:43.485317 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4eba70-b5b9-4932-bc2f-845d537cb101 admin admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:08:43.508071 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa3e9eb-2200-4a86-a274-4660b2478908 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:08:43.510785 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa3e9eb-2200-4a86-a274-4660b2478908 admin admin] REQUEST_METHOD: `PUT` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:08:43.512207 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa3e9eb-2200-4a86-a274-4660b2478908 admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:08:43.512697 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa3e9eb-2200-4a86-a274-4660b2478908 admin admin] PATH_INFO: `/v3/projects/0ae3499038574a14bbc020e8636d4076/users/2191ecb7b1a64e668b2299b37f034581/roles/b2cb2a6e818e47338ce071e6fedfc673` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:08:43.604615 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa3e9eb-2200-4a86-a274-4660b2478908 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0ae3499038574a14bbc020e8636d4076, user_id=2191ecb7b1a64e668b2299b37f034581, role_id=b2cb2a6e818e47338ce071e6fedfc673)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:08:43.619158 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa3e9eb-2200-4a86-a274-4660b2478908 admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:08:43.800632 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 920/1868] 10.222.0.46 () {60 vars in 1483 bytes} [Mon Oct 10 12:08:43 2022] PUT /identity/v3/projects/0ae3499038574a14bbc020e8636d4076/users/2191ecb7b1a64e668b2299b37f034581/roles/b2cb2a6e818e47338ce071e6fedfc673 => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:08:43.816775 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67175c80-497d-470e-a595-11fcd674d425 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:08:43.817356 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67175c80-497d-470e-a595-11fcd674d425 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:08:43.818428 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67175c80-497d-470e-a595-11fcd674d425 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:08:43.907778 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-67175c80-497d-470e-a595-11fcd674d425 None None] MFA Rules not processed for user `2191ecb7b1a64e668b2299b37f034581`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:08:43.909743 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-67175c80-497d-470e-a595-11fcd674d425 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:08:44.035492 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 949/1869] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:08:43 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 10 12:08:44.042518 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa3e9eb-2200-4a86-a274-4660b2478908 admin admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:08:44.240198 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa3e9eb-2200-4a86-a274-4660b2478908 admin admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:08:44.267858 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08237bb0-35e4-4455-a7e6-ee268408d7f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:08:44.269315 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08237bb0-35e4-4455-a7e6-ee268408d7f2 admin admin] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:08:44.270043 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08237bb0-35e4-4455-a7e6-ee268408d7f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:08:44.270524 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08237bb0-35e4-4455-a7e6-ee268408d7f2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:08:44.272258 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08237bb0-35e4-4455-a7e6-ee268408d7f2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:08:44.278719 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08237bb0-35e4-4455-a7e6-ee268408d7f2 admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:08:44.451364 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 921/1870] 10.222.0.46 () {60 vars in 1153 bytes} [Mon Oct 10 12:08:44 2022] POST /identity/v3/projects => generated 377 bytes in 409 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 12:08:44.455103 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67175c80-497d-470e-a595-11fcd674d425 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:08:44.474047 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67175c80-497d-470e-a595-11fcd674d425 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:08:44.487288 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:08:44.550509 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c3f34e-f02f-4a43-bb8e-472cd89907d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:08:44.551278 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c3f34e-f02f-4a43-bb8e-472cd89907d2 admin admin] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:08:44.551625 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c3f34e-f02f-4a43-bb8e-472cd89907d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:08:44.551883 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c3f34e-f02f-4a43-bb8e-472cd89907d2 admin admin] PATH_INFO: `/v3/users` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:08:44.552601 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c3f34e-f02f-4a43-bb8e-472cd89907d2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:08:44.554914 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c3f34e-f02f-4a43-bb8e-472cd89907d2 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:08:44.686792 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 950/1871] 10.222.0.46 () {60 vars in 1144 bytes} [Mon Oct 10 12:08:44 2022] POST /identity/v3/users => generated 289 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 12:08:44.690477 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08237bb0-35e4-4455-a7e6-ee268408d7f2 admin admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:08:44.706023 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08237bb0-35e4-4455-a7e6-ee268408d7f2 admin admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:08:44.715201 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d77ba6-01c0-4c3e-9c25-0251979eb615 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:08:44.716246 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d77ba6-01c0-4c3e-9c25-0251979eb615 admin admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:08:44.716763 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d77ba6-01c0-4c3e-9c25-0251979eb615 admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:08:44.717252 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d77ba6-01c0-4c3e-9c25-0251979eb615 admin admin] PATH_INFO: `/v3/roles` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:08:44.718839 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d77ba6-01c0-4c3e-9c25-0251979eb615 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:08:44.722120 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d77ba6-01c0-4c3e-9c25-0251979eb615 admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:08:44.746308 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 922/1872] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 10 12:08:44 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 12:08:44.753193 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c3f34e-f02f-4a43-bb8e-472cd89907d2 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:08:44.775508 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c3f34e-f02f-4a43-bb8e-472cd89907d2 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:08:44.785330 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd740a0a-ed31-4326-9c5e-8653cd3cb58e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:08:44.787282 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd740a0a-ed31-4326-9c5e-8653cd3cb58e admin admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:08:44.787959 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd740a0a-ed31-4326-9c5e-8653cd3cb58e admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:08:44.788519 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd740a0a-ed31-4326-9c5e-8653cd3cb58e admin admin] PATH_INFO: `/v3/roles` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:08:44.789821 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd740a0a-ed31-4326-9c5e-8653cd3cb58e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:08:44.794281 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd740a0a-ed31-4326-9c5e-8653cd3cb58e admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:08:44.818806 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 951/1873] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 10 12:08:44 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 12:08:44.825136 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d77ba6-01c0-4c3e-9c25-0251979eb615 admin admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:08:44.844668 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d77ba6-01c0-4c3e-9c25-0251979eb615 admin admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:08:44.851223 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21fb14e-c497-4275-9c39-bb284eb7afc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:08:44.851938 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21fb14e-c497-4275-9c39-bb284eb7afc4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:08:44.852222 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21fb14e-c497-4275-9c39-bb284eb7afc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:08:44.852503 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21fb14e-c497-4275-9c39-bb284eb7afc4 admin admin] PATH_INFO: `/v3/projects/7748cf47a4a84116a7cab84964cee9c7/users/ad3db52ac190448888050908c4057dc5/roles/b2cb2a6e818e47338ce071e6fedfc673` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:08:44.873000 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21fb14e-c497-4275-9c39-bb284eb7afc4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7748cf47a4a84116a7cab84964cee9c7, user_id=ad3db52ac190448888050908c4057dc5, role_id=b2cb2a6e818e47338ce071e6fedfc673)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:08:44.876028 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21fb14e-c497-4275-9c39-bb284eb7afc4 admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:08:44.960045 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 923/1874] 10.222.0.46 () {60 vars in 1483 bytes} [Mon Oct 10 12:08:44 2022] PUT /identity/v3/projects/7748cf47a4a84116a7cab84964cee9c7/users/ad3db52ac190448888050908c4057dc5/roles/b2cb2a6e818e47338ce071e6fedfc673 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:08:44.986771 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7689d2-35ad-4221-82f5-0ef0e66be49e None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:08:44.986771 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7689d2-35ad-4221-82f5-0ef0e66be49e None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:08:44.986771 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7689d2-35ad-4221-82f5-0ef0e66be49e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:08:45.010750 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-1f7689d2-35ad-4221-82f5-0ef0e66be49e None None] MFA Rules not processed for user `ad3db52ac190448888050908c4057dc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:08:45.012243 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-1f7689d2-35ad-4221-82f5-0ef0e66be49e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:08:45.096015 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 952/1875] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:08:44 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:08:45.102067 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21fb14e-c497-4275-9c39-bb284eb7afc4 admin admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:08:45.190654 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21fb14e-c497-4275-9c39-bb284eb7afc4 admin admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:08:45.197005 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7708a4c-18e3-40c4-904c-91b9e78cc57d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:08:45.200596 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7708a4c-18e3-40c4-904c-91b9e78cc57d admin admin] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:08:45.200596 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7708a4c-18e3-40c4-904c-91b9e78cc57d admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:08:45.200596 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7708a4c-18e3-40c4-904c-91b9e78cc57d admin admin] PATH_INFO: `/v3/projects` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:08:45.200596 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7708a4c-18e3-40c4-904c-91b9e78cc57d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:08:45.202808 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7708a4c-18e3-40c4-904c-91b9e78cc57d admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:08:45.373442 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 924/1876] 10.222.0.46 () {60 vars in 1153 bytes} [Mon Oct 10 12:08:45 2022] POST /identity/v3/projects => generated 379 bytes in 272 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 10 12:08:45.378204 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7689d2-35ad-4221-82f5-0ef0e66be49e None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:08:45.394261 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7689d2-35ad-4221-82f5-0ef0e66be49e None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:08:45.524492 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8376e346-cf75-4eca-96e4-148d84d6ea42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:08:45.525507 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8376e346-cf75-4eca-96e4-148d84d6ea42 admin admin] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:08:45.526331 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8376e346-cf75-4eca-96e4-148d84d6ea42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:08:45.526786 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8376e346-cf75-4eca-96e4-148d84d6ea42 admin admin] PATH_INFO: `/v3/users` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:08:45.527902 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8376e346-cf75-4eca-96e4-148d84d6ea42 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:08:45.533023 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8376e346-cf75-4eca-96e4-148d84d6ea42 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:08:45.641606 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 953/1877] 10.222.0.46 () {60 vars in 1144 bytes} [Mon Oct 10 12:08:45 2022] POST /identity/v3/users => generated 296 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 12:08:45.646152 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7708a4c-18e3-40c4-904c-91b9e78cc57d admin admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:08:45.657185 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7708a4c-18e3-40c4-904c-91b9e78cc57d admin admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:08:45.666272 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9c7b7a-243a-4f73-b8ea-73888c0acae2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:08:45.667242 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9c7b7a-243a-4f73-b8ea-73888c0acae2 admin admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:08:45.667616 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9c7b7a-243a-4f73-b8ea-73888c0acae2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:08:45.668134 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9c7b7a-243a-4f73-b8ea-73888c0acae2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:08:45.669557 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9c7b7a-243a-4f73-b8ea-73888c0acae2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:08:45.672100 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9c7b7a-243a-4f73-b8ea-73888c0acae2 admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:08:45.693926 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 925/1878] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 10 12:08:45 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 12:08:45.699279 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8376e346-cf75-4eca-96e4-148d84d6ea42 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:08:45.716735 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8376e346-cf75-4eca-96e4-148d84d6ea42 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:08:45.726828 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ef291d-2202-4ccc-934f-236d83f3b213 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:08:45.728348 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ef291d-2202-4ccc-934f-236d83f3b213 admin admin] REQUEST_METHOD: `PUT` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:08:45.728852 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ef291d-2202-4ccc-934f-236d83f3b213 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:08:45.729584 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ef291d-2202-4ccc-934f-236d83f3b213 admin admin] PATH_INFO: `/v3/projects/dbd7bc8bce074558bf270f9ea02f76b6/users/c738c3663e05457b80bbf2d717192f9a/roles/0d822a0674b6495fac623877a3b38025` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:08:45.734236 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:08:45.750121 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ef291d-2202-4ccc-934f-236d83f3b213 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dbd7bc8bce074558bf270f9ea02f76b6, user_id=c738c3663e05457b80bbf2d717192f9a, role_id=0d822a0674b6495fac623877a3b38025)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:08:45.753089 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ef291d-2202-4ccc-934f-236d83f3b213 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:08:45.878290 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 954/1879] 10.222.0.46 () {60 vars in 1483 bytes} [Mon Oct 10 12:08:45 2022] PUT /identity/v3/projects/dbd7bc8bce074558bf270f9ea02f76b6/users/c738c3663e05457b80bbf2d717192f9a/roles/0d822a0674b6495fac623877a3b38025 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:08:45.884441 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9c7b7a-243a-4f73-b8ea-73888c0acae2 admin admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:08:45.901622 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:08:45.908928 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9c7b7a-243a-4f73-b8ea-73888c0acae2 admin admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:08:46.053961 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e51b0f-f036-46fb-9bbd-51daf73208d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:08:46.055675 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e51b0f-f036-46fb-9bbd-51daf73208d9 admin admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:08:46.056193 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e51b0f-f036-46fb-9bbd-51daf73208d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:08:46.056670 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e51b0f-f036-46fb-9bbd-51daf73208d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:08:46.063259 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e51b0f-f036-46fb-9bbd-51daf73208d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:08:46.074404 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e51b0f-f036-46fb-9bbd-51daf73208d9 admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:08:46.103762 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 926/1880] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 10 12:08:45 2022] GET /identity/v3/roles => generated 1420 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 12:08:46.109429 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ef291d-2202-4ccc-934f-236d83f3b213 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:08:46.142798 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ef291d-2202-4ccc-934f-236d83f3b213 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:08:46.191532 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f60cbe9-f865-46e3-aa46-72096b1283a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:08:46.192555 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f60cbe9-f865-46e3-aa46-72096b1283a4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:08:46.195390 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f60cbe9-f865-46e3-aa46-72096b1283a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:08:46.198625 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f60cbe9-f865-46e3-aa46-72096b1283a4 admin admin] PATH_INFO: `/v3/projects/dbd7bc8bce074558bf270f9ea02f76b6/users/c738c3663e05457b80bbf2d717192f9a/roles/0d822a0674b6495fac623877a3b38025` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:08:46.203953 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f60cbe9-f865-46e3-aa46-72096b1283a4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dbd7bc8bce074558bf270f9ea02f76b6, user_id=c738c3663e05457b80bbf2d717192f9a, role_id=0d822a0674b6495fac623877a3b38025)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:08:46.207751 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f60cbe9-f865-46e3-aa46-72096b1283a4 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:08:46.234786 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 955/1881] 10.222.0.46 () {60 vars in 1483 bytes} [Mon Oct 10 12:08:46 2022] PUT /identity/v3/projects/dbd7bc8bce074558bf270f9ea02f76b6/users/c738c3663e05457b80bbf2d717192f9a/roles/0d822a0674b6495fac623877a3b38025 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:08:46.254055 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1297d8fb-778e-4a06-b044-f5f2f4246b68 None None] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:08:46.257478 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1297d8fb-778e-4a06-b044-f5f2f4246b68 None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:08:46.258638 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1297d8fb-778e-4a06-b044-f5f2f4246b68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:08:46.328234 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.auth.core [None req-1297d8fb-778e-4a06-b044-f5f2f4246b68 None None] MFA Rules not processed for user `c738c3663e05457b80bbf2d717192f9a`. Rule list: `[]` (Enabled: `True`). {{(pid=81676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:08:46.330354 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-1297d8fb-778e-4a06-b044-f5f2f4246b68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:08:46.494602 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 927/1882] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:08:46 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:08:46.519403 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f60cbe9-f865-46e3-aa46-72096b1283a4 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:08:46.522857 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-8f60cbe9-f865-46e3-aa46-72096b1283a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:08:46.660621 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f60cbe9-f865-46e3-aa46-72096b1283a4 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:08:46.666593 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1b604e-6a02-4516-825c-439fc3ecff44 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3ae093470484b93b23dcfae1657df9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad3772f7b6a47c28b7ba37a9316a3c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:08:46.667809 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1b604e-6a02-4516-825c-439fc3ecff44 service cinder] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:08:46.668488 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1b604e-6a02-4516-825c-439fc3ecff44 service cinder] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:08:46.669086 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1b604e-6a02-4516-825c-439fc3ecff44 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:08:46.684502 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1b604e-6a02-4516-825c-439fc3ecff44 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:08:46.686884 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1b604e-6a02-4516-825c-439fc3ecff44 service cinder] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:08:46.704575 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 956/1883] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 10 12:08:46 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:08:46.906302 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1297d8fb-778e-4a06-b044-f5f2f4246b68 None None] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:08:46.908989 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-1297d8fb-778e-4a06-b044-f5f2f4246b68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:08:46.934968 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1297d8fb-778e-4a06-b044-f5f2f4246b68 None None] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:08:46.950297 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7c1d4c-38ec-4b43-bbb5-bdecf5f861cb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3ae093470484b93b23dcfae1657df9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad3772f7b6a47c28b7ba37a9316a3c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:08:46.953970 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7c1d4c-38ec-4b43-bbb5-bdecf5f861cb service cinder] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:08:46.954591 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7c1d4c-38ec-4b43-bbb5-bdecf5f861cb service cinder] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:08:46.955564 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7c1d4c-38ec-4b43-bbb5-bdecf5f861cb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:08:47.182642 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7c1d4c-38ec-4b43-bbb5-bdecf5f861cb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:08:47.186607 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7c1d4c-38ec-4b43-bbb5-bdecf5f861cb service cinder] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:08:47.214977 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 928/1884] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 10 12:08:46 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 10 12:08:54.210445 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1b604e-6a02-4516-825c-439fc3ecff44 service cinder] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:08:54.334107 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1b604e-6a02-4516-825c-439fc3ecff44 service cinder] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:08:54.339500 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ac8adc-297e-477f-8ca1-a1ace5afe5e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:08:54.340176 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ac8adc-297e-477f-8ca1-a1ace5afe5e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:08:54.340516 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ac8adc-297e-477f-8ca1-a1ace5afe5e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:08:54.340824 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ac8adc-297e-477f-8ca1-a1ace5afe5e6 admin admin] PATH_INFO: `/v3/users/2191ecb7b1a64e668b2299b37f034581` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:08:54.342084 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ac8adc-297e-477f-8ca1-a1ace5afe5e6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2191ecb7b1a64e668b2299b37f034581)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:08:54.344172 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ac8adc-297e-477f-8ca1-a1ace5afe5e6 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:08:54.593356 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-f6ac8adc-297e-477f-8ca1-a1ace5afe5e6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2191ecb7b1a64e668b2299b37f034581'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:08:54.605634 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-f6ac8adc-297e-477f-8ca1-a1ace5afe5e6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2191ecb7b1a64e668b2299b37f034581'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:08:54.607549 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-f6ac8adc-297e-477f-8ca1-a1ace5afe5e6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2191ecb7b1a64e668b2299b37f034581'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:08:54.698944 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-f6ac8adc-297e-477f-8ca1-a1ace5afe5e6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2191ecb7b1a64e668b2299b37f034581'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:08:54.714361 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=88,keystone:INSERT=3,keystone:DELETE=9 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:08:54.717945 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-f6ac8adc-297e-477f-8ca1-a1ace5afe5e6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2191ecb7b1a64e668b2299b37f034581'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:08:54.732905 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 957/1885] 10.222.0.46 () {58 vars in 1224 bytes} [Mon Oct 10 12:08:54 2022] DELETE /identity/v3/users/2191ecb7b1a64e668b2299b37f034581 => generated 0 bytes in 523 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:08:54.737486 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e49d215-07ad-4430-b507-941771bc0acf None None] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:08:54.738517 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e49d215-07ad-4430-b507-941771bc0acf None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:08:54.738517 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e49d215-07ad-4430-b507-941771bc0acf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:08:54.761434 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.auth.core [None req-6e49d215-07ad-4430-b507-941771bc0acf None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:08:54.762800 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-6e49d215-07ad-4430-b507-941771bc0acf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:08:54.786471 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:08:54.866082 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 929/1886] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:08:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:08:54.873253 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ac8adc-297e-477f-8ca1-a1ace5afe5e6 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:08:54.875296 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-f6ac8adc-297e-477f-8ca1-a1ace5afe5e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:08:54.953371 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c8920d-9518-418f-bdc5-9447b109ce2c None None] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:08:54.955978 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c8920d-9518-418f-bdc5-9447b109ce2c None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:08:54.955978 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c8920d-9518-418f-bdc5-9447b109ce2c None None] PATH_INFO: `/` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:08:54.959765 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 930/1887] 10.222.0.46 () {54 vars in 921 bytes} [Mon Oct 10 12:08:54 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 12:08:54.980426 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82627786-1d1e-4d62-8006-012d26a3ae02 None None] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:08:54.981217 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82627786-1d1e-4d62-8006-012d26a3ae02 None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:08:54.981900 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82627786-1d1e-4d62-8006-012d26a3ae02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:08:54.989804 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ac8adc-297e-477f-8ca1-a1ace5afe5e6 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:08:55.000437 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7aa9fe-6d8e-4f5c-989a-422230f8d9ec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58b1c7c85302421cba823ace4201861f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad3772f7b6a47c28b7ba37a9316a3c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:08:55.001707 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7aa9fe-6d8e-4f5c-989a-422230f8d9ec service neutron] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:08:55.002153 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7aa9fe-6d8e-4f5c-989a-422230f8d9ec service neutron] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:08:55.002535 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7aa9fe-6d8e-4f5c-989a-422230f8d9ec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:08:55.017844 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7aa9fe-6d8e-4f5c-989a-422230f8d9ec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:08:55.019839 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7aa9fe-6d8e-4f5c-989a-422230f8d9ec service neutron] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:08:55.028098 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.auth.core [None req-82627786-1d1e-4d62-8006-012d26a3ae02 None None] MFA Rules not processed for user `8e8f9bb1776d47169ef7febb57c954cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:08:55.030937 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-82627786-1d1e-4d62-8006-012d26a3ae02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:08:55.038444 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 958/1888] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 10 12:08:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 10 12:08:55.132369 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 931/1889] 10.222.0.46 () {58 vars in 1018 bytes} [Mon Oct 10 12:08:54 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:08:55.151907 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e58f9b1-a468-45e4-977f-00b59d591302 None None] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:08:55.152245 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e58f9b1-a468-45e4-977f-00b59d591302 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:08:55.152614 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e58f9b1-a468-45e4-977f-00b59d591302 None None] PATH_INFO: `/` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:08:55.154876 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 959/1890] 10.222.0.46 () {54 vars in 961 bytes} [Mon Oct 10 12:08:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 10 12:08:55.163912 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884ab9f2-66d2-4d0b-9951-890c68dbcfa8 None None] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:08:55.165880 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884ab9f2-66d2-4d0b-9951-890c68dbcfa8 None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:08:55.166528 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884ab9f2-66d2-4d0b-9951-890c68dbcfa8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:08:55.205971 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.auth.core [None req-884ab9f2-66d2-4d0b-9951-890c68dbcfa8 None None] MFA Rules not processed for user `8e8f9bb1776d47169ef7febb57c954cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:08:55.208823 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-884ab9f2-66d2-4d0b-9951-890c68dbcfa8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:08:55.225148 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 932/1891] 10.222.0.46 () {58 vars in 1058 bytes} [Mon Oct 10 12:08:55 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 10 12:08:55.233666 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e58f9b1-a468-45e4-977f-00b59d591302 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:08:55.269875 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e58f9b1-a468-45e4-977f-00b59d591302 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:08:55.283774 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c69ca3b-765b-451e-b6a6-50f06e07d083 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e8f9bb1776d47169ef7febb57c954cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad3772f7b6a47c28b7ba37a9316a3c1', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:08:55.285134 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c69ca3b-765b-451e-b6a6-50f06e07d083 service placement] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:08:55.285134 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c69ca3b-765b-451e-b6a6-50f06e07d083 service placement] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:08:55.285134 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c69ca3b-765b-451e-b6a6-50f06e07d083 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:08:55.298323 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c69ca3b-765b-451e-b6a6-50f06e07d083 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:08:55.300090 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c69ca3b-765b-451e-b6a6-50f06e07d083 service placement] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:08:55.327208 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 960/1892] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 10 12:08:55 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 10 12:08:55.984446 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884ab9f2-66d2-4d0b-9951-890c68dbcfa8 None None] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:08:55.989828 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-884ab9f2-66d2-4d0b-9951-890c68dbcfa8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:08:56.006898 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884ab9f2-66d2-4d0b-9951-890c68dbcfa8 None None] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:08:56.012180 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51c82c5-22f5-4594-aa6a-83b8fe8aafda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:08:56.016651 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51c82c5-22f5-4594-aa6a-83b8fe8aafda admin admin] REQUEST_METHOD: `DELETE` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:08:56.016651 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51c82c5-22f5-4594-aa6a-83b8fe8aafda admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:08:56.016651 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51c82c5-22f5-4594-aa6a-83b8fe8aafda admin admin] PATH_INFO: `/v3/projects/0ae3499038574a14bbc020e8636d4076` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:08:56.016651 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51c82c5-22f5-4594-aa6a-83b8fe8aafda admin admin] RBAC: Authorizing `identity:delete_project(project_id=0ae3499038574a14bbc020e8636d4076)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:08:56.016651 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51c82c5-22f5-4594-aa6a-83b8fe8aafda admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:08:56.154620 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-b51c82c5-22f5-4594-aa6a-83b8fe8aafda admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0ae3499038574a14bbc020e8636d4076', 'request_id': 'req-b51c82c5-22f5-4594-aa6a-83b8fe8aafda'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:08:56.167567 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-b51c82c5-22f5-4594-aa6a-83b8fe8aafda admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0ae3499038574a14bbc020e8636d4076', 'request_id': 'req-b51c82c5-22f5-4594-aa6a-83b8fe8aafda'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:08:56.190492 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-b51c82c5-22f5-4594-aa6a-83b8fe8aafda admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0ae3499038574a14bbc020e8636d4076', 'request_id': 'req-b51c82c5-22f5-4594-aa6a-83b8fe8aafda'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:08:56.319692 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-b51c82c5-22f5-4594-aa6a-83b8fe8aafda admin admin] The token cache is being invalidate because project 0ae3499038574a14bbc020e8636d4076 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81676) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 10 12:08:56.320225 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-b51c82c5-22f5-4594-aa6a-83b8fe8aafda admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:08:56.321246 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-b51c82c5-22f5-4594-aa6a-83b8fe8aafda admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:08:56.326692 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 933/1893] 10.222.0.46 () {58 vars in 1233 bytes} [Mon Oct 10 12:08:55 2022] DELETE /identity/v3/projects/0ae3499038574a14bbc020e8636d4076 => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:08:56.328571 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c69ca3b-765b-451e-b6a6-50f06e07d083 service placement] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:08:56.348128 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c69ca3b-765b-451e-b6a6-50f06e07d083 service placement] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:08:56.435381 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ab1295-a43d-4423-82e1-caad96aa97e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:08:56.436039 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ab1295-a43d-4423-82e1-caad96aa97e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:08:56.436321 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ab1295-a43d-4423-82e1-caad96aa97e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:08:56.436598 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ab1295-a43d-4423-82e1-caad96aa97e3 admin admin] PATH_INFO: `/v3/users/ad3db52ac190448888050908c4057dc5` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:08:56.437942 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ab1295-a43d-4423-82e1-caad96aa97e3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ad3db52ac190448888050908c4057dc5)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:08:56.440136 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ab1295-a43d-4423-82e1-caad96aa97e3 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:08:56.625753 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51c82c5-22f5-4594-aa6a-83b8fe8aafda admin admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:08:56.627518 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-b51c82c5-22f5-4594-aa6a-83b8fe8aafda admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:08:56.643649 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51c82c5-22f5-4594-aa6a-83b8fe8aafda admin admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:08:56.649449 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c885c9b5-16a3-491e-8ce5-58a722d6496b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:08:56.650114 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c885c9b5-16a3-491e-8ce5-58a722d6496b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:08:56.650393 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c885c9b5-16a3-491e-8ce5-58a722d6496b admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:08:56.650708 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c885c9b5-16a3-491e-8ce5-58a722d6496b admin admin] PATH_INFO: `/v3/users/595183b0f0914211969d5da466dc4598` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:08:56.651956 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c885c9b5-16a3-491e-8ce5-58a722d6496b admin admin] RBAC: Authorizing `identity:delete_user(user_id=595183b0f0914211969d5da466dc4598)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:08:56.653967 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c885c9b5-16a3-491e-8ce5-58a722d6496b admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:08:56.721426 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-c6ab1295-a43d-4423-82e1-caad96aa97e3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ad3db52ac190448888050908c4057dc5'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:08:56.737876 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-c6ab1295-a43d-4423-82e1-caad96aa97e3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ad3db52ac190448888050908c4057dc5'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:08:56.738779 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-c6ab1295-a43d-4423-82e1-caad96aa97e3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ad3db52ac190448888050908c4057dc5'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:08:56.841662 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:08:56.845577 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-c6ab1295-a43d-4423-82e1-caad96aa97e3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ad3db52ac190448888050908c4057dc5'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:08:56.864972 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-c885c9b5-16a3-491e-8ce5-58a722d6496b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '595183b0f0914211969d5da466dc4598'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:08:56.874290 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-c885c9b5-16a3-491e-8ce5-58a722d6496b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '595183b0f0914211969d5da466dc4598'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:08:56.875287 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-c885c9b5-16a3-491e-8ce5-58a722d6496b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '595183b0f0914211969d5da466dc4598'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:08:56.878034 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-c6ab1295-a43d-4423-82e1-caad96aa97e3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ad3db52ac190448888050908c4057dc5'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:08:56.898213 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 961/1894] 10.222.0.46 () {58 vars in 1224 bytes} [Mon Oct 10 12:08:56 2022] DELETE /identity/v3/users/ad3db52ac190448888050908c4057dc5 => generated 0 bytes in 571 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:08:56.979702 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-c885c9b5-16a3-491e-8ce5-58a722d6496b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '595183b0f0914211969d5da466dc4598'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:08:56.999116 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-c885c9b5-16a3-491e-8ce5-58a722d6496b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '595183b0f0914211969d5da466dc4598'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:08:57.024379 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 934/1895] 10.222.0.46 () {58 vars in 1224 bytes} [Mon Oct 10 12:08:56 2022] DELETE /identity/v3/users/595183b0f0914211969d5da466dc4598 => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 10 12:08:57.302388 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ab1295-a43d-4423-82e1-caad96aa97e3 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:08:57.306096 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-c6ab1295-a43d-4423-82e1-caad96aa97e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:08:57.331578 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:08:57.424879 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ab1295-a43d-4423-82e1-caad96aa97e3 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:08:57.601789 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e4ff58-22f4-43e6-a3d5-685b8c8f6ffd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:08:57.602906 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e4ff58-22f4-43e6-a3d5-685b8c8f6ffd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:08:57.603252 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e4ff58-22f4-43e6-a3d5-685b8c8f6ffd admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:08:57.603588 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e4ff58-22f4-43e6-a3d5-685b8c8f6ffd admin admin] PATH_INFO: `/v3/projects/d17d399eda224c16a81052f5faff95b4` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:08:57.605160 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e4ff58-22f4-43e6-a3d5-685b8c8f6ffd admin admin] RBAC: Authorizing `identity:delete_project(project_id=d17d399eda224c16a81052f5faff95b4)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:08:57.607316 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e4ff58-22f4-43e6-a3d5-685b8c8f6ffd admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:08:57.764779 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-34e4ff58-22f4-43e6-a3d5-685b8c8f6ffd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd17d399eda224c16a81052f5faff95b4', 'request_id': 'req-34e4ff58-22f4-43e6-a3d5-685b8c8f6ffd'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:08:57.773577 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-34e4ff58-22f4-43e6-a3d5-685b8c8f6ffd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd17d399eda224c16a81052f5faff95b4', 'request_id': 'req-34e4ff58-22f4-43e6-a3d5-685b8c8f6ffd'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:08:57.784298 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-34e4ff58-22f4-43e6-a3d5-685b8c8f6ffd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd17d399eda224c16a81052f5faff95b4', 'request_id': 'req-34e4ff58-22f4-43e6-a3d5-685b8c8f6ffd'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:08:57.953157 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-34e4ff58-22f4-43e6-a3d5-685b8c8f6ffd admin admin] The token cache is being invalidate because project d17d399eda224c16a81052f5faff95b4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81677) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 10 12:08:57.953733 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-34e4ff58-22f4-43e6-a3d5-685b8c8f6ffd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:08:57.955067 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-34e4ff58-22f4-43e6-a3d5-685b8c8f6ffd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:08:57.957889 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 962/1896] 10.222.0.46 () {58 vars in 1233 bytes} [Mon Oct 10 12:08:57 2022] DELETE /identity/v3/projects/d17d399eda224c16a81052f5faff95b4 => generated 0 bytes in 656 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:08:57.963610 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c885c9b5-16a3-491e-8ce5-58a722d6496b admin admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:08:57.965517 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-c885c9b5-16a3-491e-8ce5-58a722d6496b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:08:57.989205 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e4ff58-22f4-43e6-a3d5-685b8c8f6ffd admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:08:57.992435 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c885c9b5-16a3-491e-8ce5-58a722d6496b admin admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:08:57.992730 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-34e4ff58-22f4-43e6-a3d5-685b8c8f6ffd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:08:58.128234 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e4ff58-22f4-43e6-a3d5-685b8c8f6ffd admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:08:58.138796 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53596743-a93e-456e-bdc8-bb7cec71e967 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:08:58.140039 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53596743-a93e-456e-bdc8-bb7cec71e967 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:08:58.140681 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53596743-a93e-456e-bdc8-bb7cec71e967 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:08:58.141435 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53596743-a93e-456e-bdc8-bb7cec71e967 admin admin] PATH_INFO: `/v3/projects/7748cf47a4a84116a7cab84964cee9c7` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:08:58.152139 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53596743-a93e-456e-bdc8-bb7cec71e967 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7748cf47a4a84116a7cab84964cee9c7)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:08:58.152384 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53596743-a93e-456e-bdc8-bb7cec71e967 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:08:58.157526 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1709072-ac1b-4cc2-b253-1f8ab4d9616e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:08:58.159644 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1709072-ac1b-4cc2-b253-1f8ab4d9616e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:08:58.159644 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1709072-ac1b-4cc2-b253-1f8ab4d9616e admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:08:58.160036 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1709072-ac1b-4cc2-b253-1f8ab4d9616e admin admin] PATH_INFO: `/v3/users/0886c8e7e1814e51a5b9b20807d33098` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:08:58.168145 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1709072-ac1b-4cc2-b253-1f8ab4d9616e admin admin] RBAC: Authorizing `identity:delete_user(user_id=0886c8e7e1814e51a5b9b20807d33098)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:08:58.173024 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1709072-ac1b-4cc2-b253-1f8ab4d9616e admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:08:58.470009 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-c1709072-ac1b-4cc2-b253-1f8ab4d9616e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0886c8e7e1814e51a5b9b20807d33098'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:08:58.480460 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-c1709072-ac1b-4cc2-b253-1f8ab4d9616e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0886c8e7e1814e51a5b9b20807d33098'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:08:58.481529 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-c1709072-ac1b-4cc2-b253-1f8ab4d9616e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0886c8e7e1814e51a5b9b20807d33098'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:08:58.491940 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-53596743-a93e-456e-bdc8-bb7cec71e967 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7748cf47a4a84116a7cab84964cee9c7', 'request_id': 'req-53596743-a93e-456e-bdc8-bb7cec71e967'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:08:58.499479 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-53596743-a93e-456e-bdc8-bb7cec71e967 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7748cf47a4a84116a7cab84964cee9c7', 'request_id': 'req-53596743-a93e-456e-bdc8-bb7cec71e967'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:08:58.508698 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-53596743-a93e-456e-bdc8-bb7cec71e967 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7748cf47a4a84116a7cab84964cee9c7', 'request_id': 'req-53596743-a93e-456e-bdc8-bb7cec71e967'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:08:58.568803 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-c1709072-ac1b-4cc2-b253-1f8ab4d9616e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0886c8e7e1814e51a5b9b20807d33098'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:08:58.569575 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:08:58.599168 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-c1709072-ac1b-4cc2-b253-1f8ab4d9616e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0886c8e7e1814e51a5b9b20807d33098'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:08:58.611606 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 935/1897] 10.222.0.46 () {58 vars in 1224 bytes} [Mon Oct 10 12:08:57 2022] DELETE /identity/v3/users/0886c8e7e1814e51a5b9b20807d33098 => generated 0 bytes in 649 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:08:58.732468 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-53596743-a93e-456e-bdc8-bb7cec71e967 admin admin] The token cache is being invalidate because project 7748cf47a4a84116a7cab84964cee9c7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81677) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 10 12:08:58.733759 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-53596743-a93e-456e-bdc8-bb7cec71e967 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:08:58.735401 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-53596743-a93e-456e-bdc8-bb7cec71e967 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:08:58.741015 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 963/1898] 10.222.0.46 () {58 vars in 1233 bytes} [Mon Oct 10 12:08:57 2022] DELETE /identity/v3/projects/7748cf47a4a84116a7cab84964cee9c7 => generated 0 bytes in 753 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:08:58.746188 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1709072-ac1b-4cc2-b253-1f8ab4d9616e admin admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:08:58.749932 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-c1709072-ac1b-4cc2-b253-1f8ab4d9616e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:08:58.847698 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:08:58.922519 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1709072-ac1b-4cc2-b253-1f8ab4d9616e admin admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:08:58.932314 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84723b6e-54da-4fc3-86d4-51a5842e048b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:08:58.936352 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84723b6e-54da-4fc3-86d4-51a5842e048b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:08:58.936352 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84723b6e-54da-4fc3-86d4-51a5842e048b admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:08:58.936352 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84723b6e-54da-4fc3-86d4-51a5842e048b admin admin] PATH_INFO: `/v3/users/c738c3663e05457b80bbf2d717192f9a` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:08:58.937545 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84723b6e-54da-4fc3-86d4-51a5842e048b admin admin] RBAC: Authorizing `identity:delete_user(user_id=c738c3663e05457b80bbf2d717192f9a)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:08:58.941298 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84723b6e-54da-4fc3-86d4-51a5842e048b admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:08:59.143302 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53596743-a93e-456e-bdc8-bb7cec71e967 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:08:59.146258 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-53596743-a93e-456e-bdc8-bb7cec71e967 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:08:59.248362 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53596743-a93e-456e-bdc8-bb7cec71e967 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:08:59.255034 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ff105d-9d0a-40a6-ab28-4ebd3d46d9db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:08:59.255801 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ff105d-9d0a-40a6-ab28-4ebd3d46d9db admin admin] REQUEST_METHOD: `DELETE` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:08:59.256300 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ff105d-9d0a-40a6-ab28-4ebd3d46d9db admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:08:59.256619 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ff105d-9d0a-40a6-ab28-4ebd3d46d9db admin admin] PATH_INFO: `/v3/projects/45f32ff86e37403abcda138fa40c8374` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:08:59.258903 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ff105d-9d0a-40a6-ab28-4ebd3d46d9db admin admin] RBAC: Authorizing `identity:delete_project(project_id=45f32ff86e37403abcda138fa40c8374)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:08:59.261012 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ff105d-9d0a-40a6-ab28-4ebd3d46d9db admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:08:59.288451 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-84723b6e-54da-4fc3-86d4-51a5842e048b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c738c3663e05457b80bbf2d717192f9a'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:08:59.341291 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-84723b6e-54da-4fc3-86d4-51a5842e048b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c738c3663e05457b80bbf2d717192f9a'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:08:59.343571 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-84723b6e-54da-4fc3-86d4-51a5842e048b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c738c3663e05457b80bbf2d717192f9a'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:08:59.500400 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-84723b6e-54da-4fc3-86d4-51a5842e048b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c738c3663e05457b80bbf2d717192f9a'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:08:59.507287 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-77ff105d-9d0a-40a6-ab28-4ebd3d46d9db admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '45f32ff86e37403abcda138fa40c8374', 'request_id': 'req-77ff105d-9d0a-40a6-ab28-4ebd3d46d9db'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:08:59.528804 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-77ff105d-9d0a-40a6-ab28-4ebd3d46d9db admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '45f32ff86e37403abcda138fa40c8374', 'request_id': 'req-77ff105d-9d0a-40a6-ab28-4ebd3d46d9db'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:08:59.543427 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-84723b6e-54da-4fc3-86d4-51a5842e048b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c738c3663e05457b80bbf2d717192f9a'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:08:59.547184 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-77ff105d-9d0a-40a6-ab28-4ebd3d46d9db admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '45f32ff86e37403abcda138fa40c8374', 'request_id': 'req-77ff105d-9d0a-40a6-ab28-4ebd3d46d9db'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:08:59.578936 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 936/1899] 10.222.0.46 () {58 vars in 1224 bytes} [Mon Oct 10 12:08:58 2022] DELETE /identity/v3/users/c738c3663e05457b80bbf2d717192f9a => generated 0 bytes in 834 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 10 12:08:59.682184 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:08:59.690274 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-77ff105d-9d0a-40a6-ab28-4ebd3d46d9db admin admin] The token cache is being invalidate because project 45f32ff86e37403abcda138fa40c8374 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81677) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 10 12:08:59.690844 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-77ff105d-9d0a-40a6-ab28-4ebd3d46d9db admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:08:59.692214 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-77ff105d-9d0a-40a6-ab28-4ebd3d46d9db admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:08:59.695055 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 964/1900] 10.222.0.46 () {58 vars in 1233 bytes} [Mon Oct 10 12:08:59 2022] DELETE /identity/v3/projects/45f32ff86e37403abcda138fa40c8374 => generated 0 bytes in 553 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 10 12:08:59.704736 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08243d30-53b4-4e4e-894a-53e7fcb9357c None None] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:08:59.705242 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08243d30-53b4-4e4e-894a-53e7fcb9357c None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:08:59.705741 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08243d30-53b4-4e4e-894a-53e7fcb9357c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:08:59.745366 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.auth.core [None req-08243d30-53b4-4e4e-894a-53e7fcb9357c None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:08:59.748337 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-08243d30-53b4-4e4e-894a-53e7fcb9357c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:08:59.874877 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 937/1901] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:08:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:08:59.879365 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3db8d0a-c24f-4f8e-b480-bad3be5bd277 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:08:59.880398 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3db8d0a-c24f-4f8e-b480-bad3be5bd277 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:08:59.880876 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3db8d0a-c24f-4f8e-b480-bad3be5bd277 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:08:59.945054 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-c3db8d0a-c24f-4f8e-b480-bad3be5bd277 None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:08:59.947344 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-c3db8d0a-c24f-4f8e-b480-bad3be5bd277 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:08:59.963782 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 965/1902] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:08:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:08:59.970249 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08243d30-53b4-4e4e-894a-53e7fcb9357c None None] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:08:59.989805 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08243d30-53b4-4e4e-894a-53e7fcb9357c None None] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:09:00.001813 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6353fe-8e7f-4e15-bf65-e3b2e6f87706 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:09:00.002649 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6353fe-8e7f-4e15-bf65-e3b2e6f87706 admin admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:09:00.003102 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6353fe-8e7f-4e15-bf65-e3b2e6f87706 admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:09:00.003551 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6353fe-8e7f-4e15-bf65-e3b2e6f87706 admin admin] PATH_INFO: `/v3/domains` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:09:00.005392 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6353fe-8e7f-4e15-bf65-e3b2e6f87706 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 12:09:00.007515 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6353fe-8e7f-4e15-bf65-e3b2e6f87706 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:09:00.011394 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6353fe-8e7f-4e15-bf65-e3b2e6f87706 admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:09:00.038752 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 938/1903] 10.222.0.46 () {58 vars in 1153 bytes} [Mon Oct 10 12:08:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 12:09:00.044351 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3db8d0a-c24f-4f8e-b480-bad3be5bd277 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:09:00.081610 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3db8d0a-c24f-4f8e-b480-bad3be5bd277 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:09:00.092636 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b320366-7d7a-4ac6-b96a-87da9fab24a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:09:00.093872 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b320366-7d7a-4ac6-b96a-87da9fab24a0 admin admin] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:09:00.094357 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b320366-7d7a-4ac6-b96a-87da9fab24a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:09:00.095573 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b320366-7d7a-4ac6-b96a-87da9fab24a0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:09:00.096955 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b320366-7d7a-4ac6-b96a-87da9fab24a0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:09:00.101306 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b320366-7d7a-4ac6-b96a-87da9fab24a0 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:09:00.257698 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 966/1904] 10.222.0.46 () {60 vars in 1153 bytes} [Mon Oct 10 12:09:00 2022] POST /identity/v3/projects => generated 383 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 12:09:00.264062 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6353fe-8e7f-4e15-bf65-e3b2e6f87706 admin admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:09:00.321856 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6353fe-8e7f-4e15-bf65-e3b2e6f87706 admin admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:09:00.421277 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b320366-7d7a-4ac6-b96a-87da9fab24a0 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:09:00.426116 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-3b320366-7d7a-4ac6-b96a-87da9fab24a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:09:00.439754 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:09:00.598257 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1198a9-19a9-44a3-be83-777afcbb6d08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:09:00.602459 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1198a9-19a9-44a3-be83-777afcbb6d08 admin admin] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:09:00.603234 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1198a9-19a9-44a3-be83-777afcbb6d08 admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:09:00.606251 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1198a9-19a9-44a3-be83-777afcbb6d08 admin admin] PATH_INFO: `/v3/users` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:09:00.609043 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1198a9-19a9-44a3-be83-777afcbb6d08 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:09:00.616464 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1198a9-19a9-44a3-be83-777afcbb6d08 admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:09:00.711277 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b320366-7d7a-4ac6-b96a-87da9fab24a0 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:09:00.720409 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd3eda1-ed34-4dd8-a5c2-112400be8b33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:09:00.721804 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd3eda1-ed34-4dd8-a5c2-112400be8b33 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:09:00.722384 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd3eda1-ed34-4dd8-a5c2-112400be8b33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:09:00.722989 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd3eda1-ed34-4dd8-a5c2-112400be8b33 admin admin] PATH_INFO: `/v3/projects/dbd7bc8bce074558bf270f9ea02f76b6` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:09:00.726337 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd3eda1-ed34-4dd8-a5c2-112400be8b33 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dbd7bc8bce074558bf270f9ea02f76b6)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:09:00.730794 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd3eda1-ed34-4dd8-a5c2-112400be8b33 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:09:00.787872 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 939/1905] 10.222.0.46 () {60 vars in 1144 bytes} [Mon Oct 10 12:09:00 2022] POST /identity/v3/users => generated 292 bytes in 525 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 12:09:00.795258 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1198a9-19a9-44a3-be83-777afcbb6d08 admin admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:09:00.824590 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1198a9-19a9-44a3-be83-777afcbb6d08 admin admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:09:00.838154 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8191437e-05ca-4a0a-82b0-aa2608900191 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:09:00.839650 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8191437e-05ca-4a0a-82b0-aa2608900191 admin admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:09:00.840398 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8191437e-05ca-4a0a-82b0-aa2608900191 admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:09:00.841279 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8191437e-05ca-4a0a-82b0-aa2608900191 admin admin] PATH_INFO: `/v3/roles` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:09:00.842783 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8191437e-05ca-4a0a-82b0-aa2608900191 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:09:00.846320 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8191437e-05ca-4a0a-82b0-aa2608900191 admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:09:00.875242 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 940/1906] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 10 12:09:00 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 12:09:00.877574 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8191437e-05ca-4a0a-82b0-aa2608900191 admin admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:09:00.901386 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8191437e-05ca-4a0a-82b0-aa2608900191 admin admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:09:00.923483 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-5cd3eda1-ed34-4dd8-a5c2-112400be8b33 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dbd7bc8bce074558bf270f9ea02f76b6', 'request_id': 'req-5cd3eda1-ed34-4dd8-a5c2-112400be8b33'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:09:00.934222 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-5cd3eda1-ed34-4dd8-a5c2-112400be8b33 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dbd7bc8bce074558bf270f9ea02f76b6', 'request_id': 'req-5cd3eda1-ed34-4dd8-a5c2-112400be8b33'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:09:00.943305 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-5cd3eda1-ed34-4dd8-a5c2-112400be8b33 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dbd7bc8bce074558bf270f9ea02f76b6', 'request_id': 'req-5cd3eda1-ed34-4dd8-a5c2-112400be8b33'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:09:01.009704 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6b0bcf-84f6-4baf-8f8a-49b71d7e1052 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:09:01.010614 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6b0bcf-84f6-4baf-8f8a-49b71d7e1052 admin admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:09:01.011124 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6b0bcf-84f6-4baf-8f8a-49b71d7e1052 admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:09:01.011605 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6b0bcf-84f6-4baf-8f8a-49b71d7e1052 admin admin] PATH_INFO: `/v3/roles` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:09:01.012823 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6b0bcf-84f6-4baf-8f8a-49b71d7e1052 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:09:01.015965 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6b0bcf-84f6-4baf-8f8a-49b71d7e1052 admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:09:01.031634 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 941/1907] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 10 12:09:00 2022] GET /identity/v3/roles => generated 1420 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 12:09:01.036670 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6b0bcf-84f6-4baf-8f8a-49b71d7e1052 admin admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:09:01.050387 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6b0bcf-84f6-4baf-8f8a-49b71d7e1052 admin admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:09:01.053872 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ebc04a-ed49-455e-bb8d-d62fd181ead7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:09:01.054575 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ebc04a-ed49-455e-bb8d-d62fd181ead7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:09:01.055016 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ebc04a-ed49-455e-bb8d-d62fd181ead7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:09:01.055168 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ebc04a-ed49-455e-bb8d-d62fd181ead7 admin admin] PATH_INFO: `/v3/projects/771f46b5f15044a9a19d1f8f047adfb8/users/e77285922e2641969b8156205afeccff/roles/b2cb2a6e818e47338ce071e6fedfc673` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:09:01.076530 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-5cd3eda1-ed34-4dd8-a5c2-112400be8b33 admin admin] The token cache is being invalidate because project dbd7bc8bce074558bf270f9ea02f76b6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81677) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 10 12:09:01.076927 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-5cd3eda1-ed34-4dd8-a5c2-112400be8b33 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:09:01.082459 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-5cd3eda1-ed34-4dd8-a5c2-112400be8b33 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:09:01.082459 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 967/1908] 10.222.0.46 () {58 vars in 1233 bytes} [Mon Oct 10 12:09:00 2022] DELETE /identity/v3/projects/dbd7bc8bce074558bf270f9ea02f76b6 => generated 0 bytes in 662 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 10 12:09:01.090057 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc23be8-f7a8-47ab-adbc-f67db4573ec3 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:09:01.090503 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc23be8-f7a8-47ab-adbc-f67db4573ec3 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:09:01.090893 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc23be8-f7a8-47ab-adbc-f67db4573ec3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:09:01.099993 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ebc04a-ed49-455e-bb8d-d62fd181ead7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=771f46b5f15044a9a19d1f8f047adfb8, user_id=e77285922e2641969b8156205afeccff, role_id=b2cb2a6e818e47338ce071e6fedfc673)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:09:01.105187 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ebc04a-ed49-455e-bb8d-d62fd181ead7 admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:09:01.128746 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-9bc23be8-f7a8-47ab-adbc-f67db4573ec3 None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:09:01.130389 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-9bc23be8-f7a8-47ab-adbc-f67db4573ec3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:09:01.149688 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 968/1909] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:09:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 10 12:09:01.164879 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd104ec0-1563-43ba-a771-32dbbd2eeb95 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:09:01.165386 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd104ec0-1563-43ba-a771-32dbbd2eeb95 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:09:01.166085 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd104ec0-1563-43ba-a771-32dbbd2eeb95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:09:01.203073 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-fd104ec0-1563-43ba-a771-32dbbd2eeb95 None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:09:01.205682 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-fd104ec0-1563-43ba-a771-32dbbd2eeb95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:09:01.220898 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 942/1910] 10.222.0.46 () {60 vars in 1483 bytes} [Mon Oct 10 12:09:01 2022] PUT /identity/v3/projects/771f46b5f15044a9a19d1f8f047adfb8/users/e77285922e2641969b8156205afeccff/roles/b2cb2a6e818e47338ce071e6fedfc673 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 10 12:09:01.241520 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bb0027-e7e4-410f-a636-8bd78ed3ee02 None None] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:09:01.242201 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bb0027-e7e4-410f-a636-8bd78ed3ee02 None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:09:01.242824 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bb0027-e7e4-410f-a636-8bd78ed3ee02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:09:01.300975 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.auth.core [None req-22bb0027-e7e4-410f-a636-8bd78ed3ee02 None None] MFA Rules not processed for user `e77285922e2641969b8156205afeccff`. Rule list: `[]` (Enabled: `True`). {{(pid=81676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:09:01.303289 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-22bb0027-e7e4-410f-a636-8bd78ed3ee02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:09:01.359920 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:09:01.384247 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 969/1911] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:09:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:09:01.390271 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd104ec0-1563-43ba-a771-32dbbd2eeb95 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:09:01.408281 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd104ec0-1563-43ba-a771-32dbbd2eeb95 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:09:01.471358 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0c11cb-dc1e-4917-ae5c-3800c2385381 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:09:01.471358 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0c11cb-dc1e-4917-ae5c-3800c2385381 admin admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:09:01.471358 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0c11cb-dc1e-4917-ae5c-3800c2385381 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:09:01.471358 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0c11cb-dc1e-4917-ae5c-3800c2385381 admin admin] PATH_INFO: `/v3/domains` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:09:01.473316 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0c11cb-dc1e-4917-ae5c-3800c2385381 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 12:09:01.473316 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0c11cb-dc1e-4917-ae5c-3800c2385381 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:09:01.473316 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0c11cb-dc1e-4917-ae5c-3800c2385381 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:09:01.474252 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 943/1912] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:09:01 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 10 12:09:01.474753 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 970/1913] 10.222.0.46 () {58 vars in 1153 bytes} [Mon Oct 10 12:09:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 12:09:01.483028 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22bb0027-e7e4-410f-a636-8bd78ed3ee02 None None] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:09:01.486761 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0c11cb-dc1e-4917-ae5c-3800c2385381 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:09:01.492041 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-8c0c11cb-dc1e-4917-ae5c-3800c2385381 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:09:01.499401 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22bb0027-e7e4-410f-a636-8bd78ed3ee02 None None] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:09:01.504569 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a3e789-ec75-452d-b346-405582f27c2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:09:01.545858 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a3e789-ec75-452d-b346-405582f27c2a admin admin] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:09:01.545858 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a3e789-ec75-452d-b346-405582f27c2a admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:09:01.545858 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a3e789-ec75-452d-b346-405582f27c2a admin admin] PATH_INFO: `/v3/projects` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:09:01.545858 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a3e789-ec75-452d-b346-405582f27c2a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:09:01.545858 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a3e789-ec75-452d-b346-405582f27c2a admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:09:01.547588 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0c11cb-dc1e-4917-ae5c-3800c2385381 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:09:01.547588 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d367183-1224-44a5-bc01-b26dc29ae04a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:09:01.547588 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d367183-1224-44a5-bc01-b26dc29ae04a admin admin] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:09:01.547588 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d367183-1224-44a5-bc01-b26dc29ae04a admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:09:01.548725 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d367183-1224-44a5-bc01-b26dc29ae04a admin admin] PATH_INFO: `/v3/projects` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:09:01.548725 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d367183-1224-44a5-bc01-b26dc29ae04a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:09:01.548725 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d367183-1224-44a5-bc01-b26dc29ae04a admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:09:01.572662 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:09:01.724461 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 944/1914] 10.222.0.46 () {60 vars in 1153 bytes} [Mon Oct 10 12:09:01 2022] POST /identity/v3/projects => generated 385 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 12:09:01.729884 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a3e789-ec75-452d-b346-405582f27c2a admin admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:09:01.733866 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 971/1915] 10.222.0.46 () {60 vars in 1153 bytes} [Mon Oct 10 12:09:01 2022] POST /identity/v3/projects => generated 385 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 12:09:01.740773 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d367183-1224-44a5-bc01-b26dc29ae04a admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:09:01.891131 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a3e789-ec75-452d-b346-405582f27c2a admin admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:09:01.927860 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d367183-1224-44a5-bc01-b26dc29ae04a admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:09:01.948655 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1aedbfb-7e5c-43b7-8e60-c4fc020b729c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:09:01.950071 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1aedbfb-7e5c-43b7-8e60-c4fc020b729c admin admin] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:09:01.950983 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1aedbfb-7e5c-43b7-8e60-c4fc020b729c admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:09:01.951547 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1aedbfb-7e5c-43b7-8e60-c4fc020b729c admin admin] PATH_INFO: `/v3/users` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:09:01.953123 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1aedbfb-7e5c-43b7-8e60-c4fc020b729c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:09:01.958964 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1aedbfb-7e5c-43b7-8e60-c4fc020b729c admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:09:02.078375 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 972/1916] 10.222.0.46 () {60 vars in 1144 bytes} [Mon Oct 10 12:09:01 2022] POST /identity/v3/users => generated 299 bytes in 339 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 12:09:02.084518 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1aedbfb-7e5c-43b7-8e60-c4fc020b729c admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:09:02.090957 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7116c0-ceab-4043-90e1-ba8ce4a265bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:09:02.095358 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7116c0-ceab-4043-90e1-ba8ce4a265bc admin admin] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:09:02.096452 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7116c0-ceab-4043-90e1-ba8ce4a265bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:09:02.097023 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7116c0-ceab-4043-90e1-ba8ce4a265bc admin admin] PATH_INFO: `/v3/users` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:09:02.106620 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7116c0-ceab-4043-90e1-ba8ce4a265bc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:09:02.107283 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1aedbfb-7e5c-43b7-8e60-c4fc020b729c admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:09:02.108643 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7116c0-ceab-4043-90e1-ba8ce4a265bc admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:09:02.115809 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0da0a1e-6423-4b31-9f55-fcb87650c437 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:09:02.121516 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0da0a1e-6423-4b31-9f55-fcb87650c437 admin admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:09:02.121516 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0da0a1e-6423-4b31-9f55-fcb87650c437 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:09:02.121516 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0da0a1e-6423-4b31-9f55-fcb87650c437 admin admin] PATH_INFO: `/v3/roles` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:09:02.122364 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0da0a1e-6423-4b31-9f55-fcb87650c437 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:09:02.131059 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0da0a1e-6423-4b31-9f55-fcb87650c437 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:09:02.153705 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 973/1917] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 10 12:09:02 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 12:09:02.160517 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0da0a1e-6423-4b31-9f55-fcb87650c437 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:09:02.181478 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0da0a1e-6423-4b31-9f55-fcb87650c437 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:09:02.190062 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e518bcb4-b2e2-4189-a1be-72555dc7a0b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:09:02.191288 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e518bcb4-b2e2-4189-a1be-72555dc7a0b6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:09:02.191956 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e518bcb4-b2e2-4189-a1be-72555dc7a0b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:09:02.192658 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e518bcb4-b2e2-4189-a1be-72555dc7a0b6 admin admin] PATH_INFO: `/v3/projects/ed1233dffa344afe926708081a99651d/users/96d72ee58d6743929e2f009e32010804/roles/0d822a0674b6495fac623877a3b38025` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:09:02.221152 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e518bcb4-b2e2-4189-a1be-72555dc7a0b6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ed1233dffa344afe926708081a99651d, user_id=96d72ee58d6743929e2f009e32010804, role_id=0d822a0674b6495fac623877a3b38025)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:09:02.225399 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e518bcb4-b2e2-4189-a1be-72555dc7a0b6 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:09:02.273521 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 945/1918] 10.222.0.46 () {60 vars in 1144 bytes} [Mon Oct 10 12:09:01 2022] POST /identity/v3/users => generated 293 bytes in 545 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 10 12:09:02.280523 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7116c0-ceab-4043-90e1-ba8ce4a265bc admin admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:09:02.299674 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7116c0-ceab-4043-90e1-ba8ce4a265bc admin admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:09:02.308762 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6da8a98-518a-4423-89ba-db9dfec4ef18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:09:02.309826 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6da8a98-518a-4423-89ba-db9dfec4ef18 admin admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:09:02.310325 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6da8a98-518a-4423-89ba-db9dfec4ef18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:09:02.310831 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6da8a98-518a-4423-89ba-db9dfec4ef18 admin admin] PATH_INFO: `/v3/roles` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:09:02.312318 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6da8a98-518a-4423-89ba-db9dfec4ef18 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:09:02.315741 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6da8a98-518a-4423-89ba-db9dfec4ef18 admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:09:02.341253 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 946/1919] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 10 12:09:02 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 10 12:09:02.347988 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6da8a98-518a-4423-89ba-db9dfec4ef18 admin admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:09:02.362131 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6da8a98-518a-4423-89ba-db9dfec4ef18 admin admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:09:02.368487 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6068c541-368b-4116-8a11-5b8bf94c095c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:09:02.369199 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6068c541-368b-4116-8a11-5b8bf94c095c admin admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:09:02.369374 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6068c541-368b-4116-8a11-5b8bf94c095c admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:09:02.369556 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6068c541-368b-4116-8a11-5b8bf94c095c admin admin] PATH_INFO: `/v3/roles` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:09:02.370225 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6068c541-368b-4116-8a11-5b8bf94c095c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:09:02.372196 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6068c541-368b-4116-8a11-5b8bf94c095c admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:09:02.391671 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 947/1920] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 10 12:09:02 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 12:09:02.392822 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6068c541-368b-4116-8a11-5b8bf94c095c admin admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:09:02.394344 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 974/1921] 10.222.0.46 () {60 vars in 1483 bytes} [Mon Oct 10 12:09:02 2022] PUT /identity/v3/projects/ed1233dffa344afe926708081a99651d/users/96d72ee58d6743929e2f009e32010804/roles/0d822a0674b6495fac623877a3b38025 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:09:02.403359 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e518bcb4-b2e2-4189-a1be-72555dc7a0b6 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:09:02.411408 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6068c541-368b-4116-8a11-5b8bf94c095c admin admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:09:02.611105 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e518bcb4-b2e2-4189-a1be-72555dc7a0b6 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:09:02.625882 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc89603-a288-4ccd-ac3b-3a2a26edbc27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:09:02.628029 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc89603-a288-4ccd-ac3b-3a2a26edbc27 admin admin] REQUEST_METHOD: `PUT` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:09:02.628826 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc89603-a288-4ccd-ac3b-3a2a26edbc27 admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:09:02.631952 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc89603-a288-4ccd-ac3b-3a2a26edbc27 admin admin] PATH_INFO: `/v3/projects/3386a2e61eca426983604c98af697463/users/c947e76d727046b9ae485331a1e4211d/roles/b2cb2a6e818e47338ce071e6fedfc673` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:09:02.645961 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62d3449-737f-4804-9ef4-132561960926 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:09:02.648229 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62d3449-737f-4804-9ef4-132561960926 admin admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:09:02.648799 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62d3449-737f-4804-9ef4-132561960926 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:09:02.649588 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62d3449-737f-4804-9ef4-132561960926 admin admin] PATH_INFO: `/v3/roles` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:09:02.651633 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62d3449-737f-4804-9ef4-132561960926 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:09:02.657122 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62d3449-737f-4804-9ef4-132561960926 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:09:02.665398 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:09:02.673636 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc89603-a288-4ccd-ac3b-3a2a26edbc27 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3386a2e61eca426983604c98af697463, user_id=c947e76d727046b9ae485331a1e4211d, role_id=b2cb2a6e818e47338ce071e6fedfc673)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:09:02.679760 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc89603-a288-4ccd-ac3b-3a2a26edbc27 admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:09:02.707439 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 975/1922] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 10 12:09:02 2022] GET /identity/v3/roles => generated 1420 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 12:09:02.711760 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62d3449-737f-4804-9ef4-132561960926 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:09:02.762810 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62d3449-737f-4804-9ef4-132561960926 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:09:02.782325 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d3622a-9788-431f-b4ab-d306944d905e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:09:02.789960 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d3622a-9788-431f-b4ab-d306944d905e admin admin] REQUEST_METHOD: `PUT` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:09:02.793179 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d3622a-9788-431f-b4ab-d306944d905e admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:09:02.793179 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d3622a-9788-431f-b4ab-d306944d905e admin admin] PATH_INFO: `/v3/projects/ed1233dffa344afe926708081a99651d/users/96d72ee58d6743929e2f009e32010804/roles/0d822a0674b6495fac623877a3b38025` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:09:02.799848 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d3622a-9788-431f-b4ab-d306944d905e admin admin] RBAC: Authorizing `identity:create_grant(project_id=ed1233dffa344afe926708081a99651d, user_id=96d72ee58d6743929e2f009e32010804, role_id=0d822a0674b6495fac623877a3b38025)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:09:02.805591 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d3622a-9788-431f-b4ab-d306944d905e admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:09:02.809900 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 948/1923] 10.222.0.46 () {60 vars in 1483 bytes} [Mon Oct 10 12:09:02 2022] PUT /identity/v3/projects/3386a2e61eca426983604c98af697463/users/c947e76d727046b9ae485331a1e4211d/roles/b2cb2a6e818e47338ce071e6fedfc673 => generated 0 bytes in 418 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:09:02.822599 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:09:02.825206 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf10f71-bca9-4bc3-8577-8d519746298f None None] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:09:02.825206 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf10f71-bca9-4bc3-8577-8d519746298f None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:09:02.825568 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf10f71-bca9-4bc3-8577-8d519746298f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:09:02.850712 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 976/1924] 10.222.0.46 () {60 vars in 1483 bytes} [Mon Oct 10 12:09:02 2022] PUT /identity/v3/projects/ed1233dffa344afe926708081a99651d/users/96d72ee58d6743929e2f009e32010804/roles/0d822a0674b6495fac623877a3b38025 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:09:02.859714 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8025453f-3d4f-49c7-aecc-c52a09eec36a None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:09:02.862098 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8025453f-3d4f-49c7-aecc-c52a09eec36a None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:09:02.862098 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8025453f-3d4f-49c7-aecc-c52a09eec36a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:09:02.928365 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-8025453f-3d4f-49c7-aecc-c52a09eec36a None None] MFA Rules not processed for user `96d72ee58d6743929e2f009e32010804`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:09:02.944198 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-8025453f-3d4f-49c7-aecc-c52a09eec36a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:09:02.996267 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.auth.core [None req-ddf10f71-bca9-4bc3-8577-8d519746298f None None] MFA Rules not processed for user `c947e76d727046b9ae485331a1e4211d`. Rule list: `[]` (Enabled: `True`). {{(pid=81676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:09:02.999310 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-ddf10f71-bca9-4bc3-8577-8d519746298f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:09:03.159293 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 977/1925] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:09:02 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 301 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:09:03.175326 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8025453f-3d4f-49c7-aecc-c52a09eec36a None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:09:03.176766 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-8025453f-3d4f-49c7-aecc-c52a09eec36a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:09:03.213199 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 949/1926] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:09:02 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 391 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:09:03.244718 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf10f71-bca9-4bc3-8577-8d519746298f None None] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:09:03.249160 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-ddf10f71-bca9-4bc3-8577-8d519746298f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:09:03.341586 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8025453f-3d4f-49c7-aecc-c52a09eec36a None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:09:03.349876 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b0ec87-28b3-45f4-b1ef-2d1b218c396b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3ae093470484b93b23dcfae1657df9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad3772f7b6a47c28b7ba37a9316a3c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:09:03.351007 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b0ec87-28b3-45f4-b1ef-2d1b218c396b service cinder] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:09:03.351491 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b0ec87-28b3-45f4-b1ef-2d1b218c396b service cinder] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:09:03.352151 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b0ec87-28b3-45f4-b1ef-2d1b218c396b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:09:03.357256 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf10f71-bca9-4bc3-8577-8d519746298f None None] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:09:03.373972 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95a9632-ee94-494e-8a73-74ed582d34b0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3ae093470484b93b23dcfae1657df9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad3772f7b6a47c28b7ba37a9316a3c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:09:03.380984 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95a9632-ee94-494e-8a73-74ed582d34b0 service cinder] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:09:03.380984 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95a9632-ee94-494e-8a73-74ed582d34b0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:09:03.380984 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95a9632-ee94-494e-8a73-74ed582d34b0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:09:03.402185 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95a9632-ee94-494e-8a73-74ed582d34b0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:09:03.403755 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95a9632-ee94-494e-8a73-74ed582d34b0 service cinder] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:09:03.427774 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 950/1927] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 10 12:09:03 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 10 12:09:03.516548 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b0ec87-28b3-45f4-b1ef-2d1b218c396b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:09:03.519706 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b0ec87-28b3-45f4-b1ef-2d1b218c396b service cinder] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:09:03.555370 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 978/1928] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 10 12:09:03 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:09:13.422557 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=61,keystone:INSERT=1 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:09:13.531341 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=83 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:09:15.731672 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95a9632-ee94-494e-8a73-74ed582d34b0 service cinder] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:09:15.733115 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-b95a9632-ee94-494e-8a73-74ed582d34b0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:09:15.833225 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95a9632-ee94-494e-8a73-74ed582d34b0 service cinder] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:09:15.838338 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a36a23-6f03-4c87-8bd6-d831d7628a4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:09:15.839159 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a36a23-6f03-4c87-8bd6-d831d7628a4f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:09:15.839572 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a36a23-6f03-4c87-8bd6-d831d7628a4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:09:15.839853 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a36a23-6f03-4c87-8bd6-d831d7628a4f admin admin] PATH_INFO: `/v3/users/2c1cd9c6ca5b4783bbf797ae125a9e79` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:09:15.840999 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a36a23-6f03-4c87-8bd6-d831d7628a4f admin admin] RBAC: Authorizing `identity:delete_user(user_id=2c1cd9c6ca5b4783bbf797ae125a9e79)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:09:15.843064 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a36a23-6f03-4c87-8bd6-d831d7628a4f admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:09:16.106316 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-18a36a23-6f03-4c87-8bd6-d831d7628a4f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2c1cd9c6ca5b4783bbf797ae125a9e79'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:09:16.115766 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-18a36a23-6f03-4c87-8bd6-d831d7628a4f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2c1cd9c6ca5b4783bbf797ae125a9e79'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:09:16.116537 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-18a36a23-6f03-4c87-8bd6-d831d7628a4f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2c1cd9c6ca5b4783bbf797ae125a9e79'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:09:16.212993 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-18a36a23-6f03-4c87-8bd6-d831d7628a4f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2c1cd9c6ca5b4783bbf797ae125a9e79'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:09:16.228410 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-18a36a23-6f03-4c87-8bd6-d831d7628a4f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2c1cd9c6ca5b4783bbf797ae125a9e79'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:09:16.234470 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 951/1929] 10.222.0.46 () {58 vars in 1224 bytes} [Mon Oct 10 12:09:15 2022] DELETE /identity/v3/users/2c1cd9c6ca5b4783bbf797ae125a9e79 => generated 0 bytes in 503 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:09:16.238965 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82a1aa6-fff0-4bd3-882e-1f5ad6879f1f None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:09:16.239461 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82a1aa6-fff0-4bd3-882e-1f5ad6879f1f None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:09:16.239551 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82a1aa6-fff0-4bd3-882e-1f5ad6879f1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:09:16.259879 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-e82a1aa6-fff0-4bd3-882e-1f5ad6879f1f None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:09:16.261072 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-e82a1aa6-fff0-4bd3-882e-1f5ad6879f1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:09:16.327014 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 979/1930] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:09:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:09:16.337675 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a36a23-6f03-4c87-8bd6-d831d7628a4f admin admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:09:16.339211 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-18a36a23-6f03-4c87-8bd6-d831d7628a4f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:09:16.408089 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a36a23-6f03-4c87-8bd6-d831d7628a4f admin admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:09:16.414181 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70449f5-95c7-488d-9b86-4ffa248fb450 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58b1c7c85302421cba823ace4201861f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad3772f7b6a47c28b7ba37a9316a3c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:09:16.414854 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70449f5-95c7-488d-9b86-4ffa248fb450 service neutron] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:09:16.415141 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70449f5-95c7-488d-9b86-4ffa248fb450 service neutron] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:09:16.415514 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70449f5-95c7-488d-9b86-4ffa248fb450 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:09:16.425584 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70449f5-95c7-488d-9b86-4ffa248fb450 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:09:16.427198 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70449f5-95c7-488d-9b86-4ffa248fb450 service neutron] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:09:16.437766 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 952/1931] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 10 12:09:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:09:17.412431 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82a1aa6-fff0-4bd3-882e-1f5ad6879f1f None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:09:17.413900 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-e82a1aa6-fff0-4bd3-882e-1f5ad6879f1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:09:17.426758 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82a1aa6-fff0-4bd3-882e-1f5ad6879f1f None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:09:17.431638 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61131c4-e0f7-4afe-818e-007fd84a94cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:09:17.432890 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61131c4-e0f7-4afe-818e-007fd84a94cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:09:17.432890 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61131c4-e0f7-4afe-818e-007fd84a94cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:09:17.432890 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61131c4-e0f7-4afe-818e-007fd84a94cb admin admin] PATH_INFO: `/v3/projects/d80eb37d6d7a4475bcdb1594514faba1` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:09:17.434043 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61131c4-e0f7-4afe-818e-007fd84a94cb admin admin] RBAC: Authorizing `identity:delete_project(project_id=d80eb37d6d7a4475bcdb1594514faba1)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:09:17.436001 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61131c4-e0f7-4afe-818e-007fd84a94cb admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:09:17.614708 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-b61131c4-e0f7-4afe-818e-007fd84a94cb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd80eb37d6d7a4475bcdb1594514faba1', 'request_id': 'req-b61131c4-e0f7-4afe-818e-007fd84a94cb'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:09:17.622091 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-b61131c4-e0f7-4afe-818e-007fd84a94cb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd80eb37d6d7a4475bcdb1594514faba1', 'request_id': 'req-b61131c4-e0f7-4afe-818e-007fd84a94cb'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:09:17.635006 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-b61131c4-e0f7-4afe-818e-007fd84a94cb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd80eb37d6d7a4475bcdb1594514faba1', 'request_id': 'req-b61131c4-e0f7-4afe-818e-007fd84a94cb'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:09:17.773668 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-b61131c4-e0f7-4afe-818e-007fd84a94cb admin admin] The token cache is being invalidate because project d80eb37d6d7a4475bcdb1594514faba1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81677) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 10 12:09:17.774083 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-b61131c4-e0f7-4afe-818e-007fd84a94cb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:09:17.775085 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-b61131c4-e0f7-4afe-818e-007fd84a94cb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:09:17.777755 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 980/1932] 10.222.0.46 () {58 vars in 1233 bytes} [Mon Oct 10 12:09:17 2022] DELETE /identity/v3/projects/d80eb37d6d7a4475bcdb1594514faba1 => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 10 12:09:17.786313 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7a2f59-5d5c-42e7-88d0-b89df09826f3 None None] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:09:17.787132 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7a2f59-5d5c-42e7-88d0-b89df09826f3 None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:09:17.787413 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7a2f59-5d5c-42e7-88d0-b89df09826f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:09:17.831154 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.auth.core [None req-0c7a2f59-5d5c-42e7-88d0-b89df09826f3 None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:09:17.831154 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-0c7a2f59-5d5c-42e7-88d0-b89df09826f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:09:17.868331 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:09:17.938725 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 953/1933] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:09:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:09:17.971569 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f71a18-1422-4a31-8678-4a78e2e311a8 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:09:17.971754 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f71a18-1422-4a31-8678-4a78e2e311a8 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:09:17.971952 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f71a18-1422-4a31-8678-4a78e2e311a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:09:17.995038 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-74f71a18-1422-4a31-8678-4a78e2e311a8 None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:09:17.996193 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-74f71a18-1422-4a31-8678-4a78e2e311a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:09:18.001462 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7a2f59-5d5c-42e7-88d0-b89df09826f3 None None] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:09:18.003132 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-0c7a2f59-5d5c-42e7-88d0-b89df09826f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:09:18.021327 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 981/1934] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:09:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:09:18.023489 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f71a18-1422-4a31-8678-4a78e2e311a8 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:09:18.043367 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f71a18-1422-4a31-8678-4a78e2e311a8 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:09:18.050049 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d679b08e-4774-471f-9b92-632bc1708803 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:09:18.050804 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d679b08e-4774-471f-9b92-632bc1708803 admin admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:09:18.051184 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d679b08e-4774-471f-9b92-632bc1708803 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:09:18.057560 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d679b08e-4774-471f-9b92-632bc1708803 admin admin] PATH_INFO: `/v3/domains` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:09:18.057560 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d679b08e-4774-471f-9b92-632bc1708803 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 12:09:18.057560 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d679b08e-4774-471f-9b92-632bc1708803 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:09:18.057560 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d679b08e-4774-471f-9b92-632bc1708803 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:09:18.074457 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 982/1935] 10.222.0.46 () {58 vars in 1153 bytes} [Mon Oct 10 12:09:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 12:09:18.075442 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d679b08e-4774-471f-9b92-632bc1708803 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:09:18.092141 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d679b08e-4774-471f-9b92-632bc1708803 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:09:18.098162 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162c0506-6257-4587-b199-beab13e4a9fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:09:18.098934 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162c0506-6257-4587-b199-beab13e4a9fc admin admin] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:09:18.099313 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162c0506-6257-4587-b199-beab13e4a9fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:09:18.099662 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162c0506-6257-4587-b199-beab13e4a9fc admin admin] PATH_INFO: `/v3/projects` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:09:18.100426 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162c0506-6257-4587-b199-beab13e4a9fc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:09:18.102884 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162c0506-6257-4587-b199-beab13e4a9fc admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:09:18.111614 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7a2f59-5d5c-42e7-88d0-b89df09826f3 None None] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:09:18.115996 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9145d675-e4db-4961-8ae1-6df9b8d4bd11 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3ae093470484b93b23dcfae1657df9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad3772f7b6a47c28b7ba37a9316a3c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:09:18.116626 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9145d675-e4db-4961-8ae1-6df9b8d4bd11 service cinder] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:09:18.117324 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9145d675-e4db-4961-8ae1-6df9b8d4bd11 service cinder] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:09:18.117691 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9145d675-e4db-4961-8ae1-6df9b8d4bd11 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:09:18.119744 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-9145d675-e4db-4961-8ae1-6df9b8d4bd11 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:09:18.200608 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 983/1936] 10.222.0.46 () {60 vars in 1153 bytes} [Mon Oct 10 12:09:18 2022] POST /identity/v3/projects => generated 375 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 12:09:18.204971 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162c0506-6257-4587-b199-beab13e4a9fc admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:09:18.231517 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9145d675-e4db-4961-8ae1-6df9b8d4bd11 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:09:18.233032 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9145d675-e4db-4961-8ae1-6df9b8d4bd11 service cinder] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:09:18.251093 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 954/1937] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 10 12:09:18 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 250 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:09:18.277122 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:09:18.297851 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162c0506-6257-4587-b199-beab13e4a9fc admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:09:18.302832 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5675b9-1c48-435a-ad95-207ed23814cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:09:18.303465 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5675b9-1c48-435a-ad95-207ed23814cf admin admin] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:09:18.303762 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5675b9-1c48-435a-ad95-207ed23814cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:09:18.304024 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5675b9-1c48-435a-ad95-207ed23814cf admin admin] PATH_INFO: `/v3/users` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:09:18.304653 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5675b9-1c48-435a-ad95-207ed23814cf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:09:18.306588 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5675b9-1c48-435a-ad95-207ed23814cf admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:09:18.420590 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 984/1938] 10.222.0.46 () {60 vars in 1144 bytes} [Mon Oct 10 12:09:18 2022] POST /identity/v3/users => generated 288 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 12:09:18.425657 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9145d675-e4db-4961-8ae1-6df9b8d4bd11 service cinder] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:09:18.437192 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9145d675-e4db-4961-8ae1-6df9b8d4bd11 service cinder] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:09:18.442628 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e4808e-17ea-420e-bc31-b748fc26dfa2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:09:18.443272 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e4808e-17ea-420e-bc31-b748fc26dfa2 admin admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:09:18.443549 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e4808e-17ea-420e-bc31-b748fc26dfa2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:09:18.443818 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e4808e-17ea-420e-bc31-b748fc26dfa2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:09:18.444486 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e4808e-17ea-420e-bc31-b748fc26dfa2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:09:18.446373 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e4808e-17ea-420e-bc31-b748fc26dfa2 admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:09:18.465806 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 955/1939] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 10 12:09:18 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 12:09:18.472286 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5675b9-1c48-435a-ad95-207ed23814cf admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:09:18.490396 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5675b9-1c48-435a-ad95-207ed23814cf admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:09:18.497569 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0239b0-7185-47dc-8805-dc83e77491cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:09:18.498196 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0239b0-7185-47dc-8805-dc83e77491cb admin admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:09:18.498556 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0239b0-7185-47dc-8805-dc83e77491cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:09:18.498882 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0239b0-7185-47dc-8805-dc83e77491cb admin admin] PATH_INFO: `/v3/roles` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:09:18.499556 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0239b0-7185-47dc-8805-dc83e77491cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:09:18.501297 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0239b0-7185-47dc-8805-dc83e77491cb admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:09:18.513044 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 985/1940] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 10 12:09:18 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 12:09:18.517646 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e4808e-17ea-420e-bc31-b748fc26dfa2 admin admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:09:18.528285 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e4808e-17ea-420e-bc31-b748fc26dfa2 admin admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:09:18.533424 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4083b6c6-88dc-4a31-b907-aa836bdf512a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:09:18.534097 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4083b6c6-88dc-4a31-b907-aa836bdf512a admin admin] REQUEST_METHOD: `PUT` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:09:18.534386 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4083b6c6-88dc-4a31-b907-aa836bdf512a admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:09:18.534750 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4083b6c6-88dc-4a31-b907-aa836bdf512a admin admin] PATH_INFO: `/v3/projects/ef94f5b81a1e45b8bb56085399ce0f4c/users/b2815f06069a4d7bba28ff8a2be4d5d8/roles/b2cb2a6e818e47338ce071e6fedfc673` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:09:18.552098 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4083b6c6-88dc-4a31-b907-aa836bdf512a admin admin] RBAC: Authorizing `identity:create_grant(project_id=ef94f5b81a1e45b8bb56085399ce0f4c, user_id=b2815f06069a4d7bba28ff8a2be4d5d8, role_id=b2cb2a6e818e47338ce071e6fedfc673)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:09:18.554908 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4083b6c6-88dc-4a31-b907-aa836bdf512a admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:09:18.715695 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 956/1941] 10.222.0.46 () {60 vars in 1483 bytes} [Mon Oct 10 12:09:18 2022] PUT /identity/v3/projects/ef94f5b81a1e45b8bb56085399ce0f4c/users/b2815f06069a4d7bba28ff8a2be4d5d8/roles/b2cb2a6e818e47338ce071e6fedfc673 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:09:18.735279 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9228372b-0eac-45c1-858b-1334fc252cc5 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:09:18.735787 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9228372b-0eac-45c1-858b-1334fc252cc5 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:09:18.736277 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9228372b-0eac-45c1-858b-1334fc252cc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:09:18.783829 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-9228372b-0eac-45c1-858b-1334fc252cc5 None None] MFA Rules not processed for user `b2815f06069a4d7bba28ff8a2be4d5d8`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:09:18.784706 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-9228372b-0eac-45c1-858b-1334fc252cc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:09:18.857933 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 986/1942] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:09:18 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:09:18.864650 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4083b6c6-88dc-4a31-b907-aa836bdf512a admin admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:09:18.954007 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:09:18.993928 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4083b6c6-88dc-4a31-b907-aa836bdf512a admin admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:09:19.002209 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a17bd6-2db7-40a0-b8e2-14216d0adf9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:09:19.003205 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a17bd6-2db7-40a0-b8e2-14216d0adf9e admin admin] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:09:19.003712 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a17bd6-2db7-40a0-b8e2-14216d0adf9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:09:19.004247 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a17bd6-2db7-40a0-b8e2-14216d0adf9e admin admin] PATH_INFO: `/v3/projects` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:09:19.005440 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a17bd6-2db7-40a0-b8e2-14216d0adf9e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:09:19.008673 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a17bd6-2db7-40a0-b8e2-14216d0adf9e admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:09:19.124039 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 957/1943] 10.222.0.46 () {60 vars in 1153 bytes} [Mon Oct 10 12:09:18 2022] POST /identity/v3/projects => generated 371 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 12:09:19.128434 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9228372b-0eac-45c1-858b-1334fc252cc5 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:09:19.140237 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9228372b-0eac-45c1-858b-1334fc252cc5 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:09:19.233958 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f7d7cc-c6ee-40e3-a671-36f8e99f6668 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:09:19.234591 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f7d7cc-c6ee-40e3-a671-36f8e99f6668 admin admin] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:09:19.237956 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f7d7cc-c6ee-40e3-a671-36f8e99f6668 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:09:19.237956 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f7d7cc-c6ee-40e3-a671-36f8e99f6668 admin admin] PATH_INFO: `/v3/users` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:09:19.237956 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f7d7cc-c6ee-40e3-a671-36f8e99f6668 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:09:19.238332 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f7d7cc-c6ee-40e3-a671-36f8e99f6668 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:09:19.332736 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 987/1944] 10.222.0.46 () {60 vars in 1144 bytes} [Mon Oct 10 12:09:19 2022] POST /identity/v3/users => generated 292 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 12:09:19.338401 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a17bd6-2db7-40a0-b8e2-14216d0adf9e admin admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:09:19.363544 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a17bd6-2db7-40a0-b8e2-14216d0adf9e admin admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:09:19.375472 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba885e8-00bb-457a-b656-f49cc8e8ef4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:09:19.376222 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba885e8-00bb-457a-b656-f49cc8e8ef4c admin admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:09:19.376397 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba885e8-00bb-457a-b656-f49cc8e8ef4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:09:19.376671 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba885e8-00bb-457a-b656-f49cc8e8ef4c admin admin] PATH_INFO: `/v3/roles` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:09:19.377338 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba885e8-00bb-457a-b656-f49cc8e8ef4c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:09:19.379340 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba885e8-00bb-457a-b656-f49cc8e8ef4c admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:09:19.398044 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 958/1945] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 10 12:09:19 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 10 12:09:19.405001 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f7d7cc-c6ee-40e3-a671-36f8e99f6668 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:09:19.432182 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f7d7cc-c6ee-40e3-a671-36f8e99f6668 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:09:19.442037 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9442285f-b173-443f-86da-171dd8e771e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:09:19.443482 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9442285f-b173-443f-86da-171dd8e771e1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:09:19.443977 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9442285f-b173-443f-86da-171dd8e771e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:09:19.448227 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9442285f-b173-443f-86da-171dd8e771e1 admin admin] PATH_INFO: `/v3/projects/eab252b092424f8b868913277c9b66f9/users/1f198cb8e6714a619d7cbd985d3cace1/roles/0d822a0674b6495fac623877a3b38025` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:09:19.485567 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9442285f-b173-443f-86da-171dd8e771e1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eab252b092424f8b868913277c9b66f9, user_id=1f198cb8e6714a619d7cbd985d3cace1, role_id=0d822a0674b6495fac623877a3b38025)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:09:19.494038 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9442285f-b173-443f-86da-171dd8e771e1 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:09:19.601488 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 988/1946] 10.222.0.46 () {60 vars in 1483 bytes} [Mon Oct 10 12:09:19 2022] PUT /identity/v3/projects/eab252b092424f8b868913277c9b66f9/users/1f198cb8e6714a619d7cbd985d3cace1/roles/0d822a0674b6495fac623877a3b38025 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:09:19.602785 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba885e8-00bb-457a-b656-f49cc8e8ef4c admin admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:09:19.615237 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba885e8-00bb-457a-b656-f49cc8e8ef4c admin admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:09:19.678421 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079b0e04-135d-452d-ba24-71c4dcc1d7b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:09:19.679090 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079b0e04-135d-452d-ba24-71c4dcc1d7b9 admin admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:09:19.679404 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079b0e04-135d-452d-ba24-71c4dcc1d7b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:09:19.679682 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079b0e04-135d-452d-ba24-71c4dcc1d7b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:09:19.680413 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079b0e04-135d-452d-ba24-71c4dcc1d7b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:09:19.681982 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079b0e04-135d-452d-ba24-71c4dcc1d7b9 admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:09:19.693095 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 959/1947] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 10 12:09:19 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 10 12:09:19.697961 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9442285f-b173-443f-86da-171dd8e771e1 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:09:19.709842 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9442285f-b173-443f-86da-171dd8e771e1 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:09:19.714984 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a52147-d96b-4dc1-86ed-b19e80ba0af0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:09:19.715743 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a52147-d96b-4dc1-86ed-b19e80ba0af0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:09:19.716066 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a52147-d96b-4dc1-86ed-b19e80ba0af0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:09:19.716441 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a52147-d96b-4dc1-86ed-b19e80ba0af0 admin admin] PATH_INFO: `/v3/projects/eab252b092424f8b868913277c9b66f9/users/1f198cb8e6714a619d7cbd985d3cace1/roles/0d822a0674b6495fac623877a3b38025` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:09:19.718829 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a52147-d96b-4dc1-86ed-b19e80ba0af0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eab252b092424f8b868913277c9b66f9, user_id=1f198cb8e6714a619d7cbd985d3cace1, role_id=0d822a0674b6495fac623877a3b38025)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:09:19.721545 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a52147-d96b-4dc1-86ed-b19e80ba0af0 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:09:19.733074 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 989/1948] 10.222.0.46 () {60 vars in 1483 bytes} [Mon Oct 10 12:09:19 2022] PUT /identity/v3/projects/eab252b092424f8b868913277c9b66f9/users/1f198cb8e6714a619d7cbd985d3cace1/roles/0d822a0674b6495fac623877a3b38025 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:09:19.744518 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140453c7-38dd-44fc-9ee3-0639fd0199d4 None None] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:09:19.744817 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140453c7-38dd-44fc-9ee3-0639fd0199d4 None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:09:19.745106 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140453c7-38dd-44fc-9ee3-0639fd0199d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:09:19.779001 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.auth.core [None req-140453c7-38dd-44fc-9ee3-0639fd0199d4 None None] MFA Rules not processed for user `1f198cb8e6714a619d7cbd985d3cace1`. Rule list: `[]` (Enabled: `True`). {{(pid=81676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:09:19.780156 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-140453c7-38dd-44fc-9ee3-0639fd0199d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:09:19.860211 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 960/1949] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:09:19 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:09:19.873152 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a52147-d96b-4dc1-86ed-b19e80ba0af0 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:09:19.877747 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:09:19.958837 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a52147-d96b-4dc1-86ed-b19e80ba0af0 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:09:19.963039 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea0581f-ae84-4692-ab35-68cc25213d21 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3ae093470484b93b23dcfae1657df9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad3772f7b6a47c28b7ba37a9316a3c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:09:19.963666 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea0581f-ae84-4692-ab35-68cc25213d21 service cinder] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:09:19.963978 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea0581f-ae84-4692-ab35-68cc25213d21 service cinder] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:09:19.964300 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea0581f-ae84-4692-ab35-68cc25213d21 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:09:20.037972 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140453c7-38dd-44fc-9ee3-0639fd0199d4 None None] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:09:20.042734 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-140453c7-38dd-44fc-9ee3-0639fd0199d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:09:20.050799 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea0581f-ae84-4692-ab35-68cc25213d21 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:09:20.053601 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea0581f-ae84-4692-ab35-68cc25213d21 service cinder] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:09:20.076114 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 990/1950] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 10 12:09:19 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 10 12:09:20.088906 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:09:20.139888 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140453c7-38dd-44fc-9ee3-0639fd0199d4 None None] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:09:20.145738 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf01e3d4-c402-4e59-8baa-08d77a20deda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:09:20.146465 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf01e3d4-c402-4e59-8baa-08d77a20deda admin admin] REQUEST_METHOD: `DELETE` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:09:20.146780 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf01e3d4-c402-4e59-8baa-08d77a20deda admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:09:20.147065 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf01e3d4-c402-4e59-8baa-08d77a20deda admin admin] PATH_INFO: `/v3/users/c947e76d727046b9ae485331a1e4211d` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:09:20.148403 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf01e3d4-c402-4e59-8baa-08d77a20deda admin admin] RBAC: Authorizing `identity:delete_user(user_id=c947e76d727046b9ae485331a1e4211d)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:09:20.150395 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf01e3d4-c402-4e59-8baa-08d77a20deda admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:09:20.418235 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-bf01e3d4-c402-4e59-8baa-08d77a20deda admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c947e76d727046b9ae485331a1e4211d'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:09:20.433914 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-bf01e3d4-c402-4e59-8baa-08d77a20deda admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c947e76d727046b9ae485331a1e4211d'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:09:20.434881 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-bf01e3d4-c402-4e59-8baa-08d77a20deda admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c947e76d727046b9ae485331a1e4211d'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:09:20.532765 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-bf01e3d4-c402-4e59-8baa-08d77a20deda admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c947e76d727046b9ae485331a1e4211d'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:09:20.551019 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-bf01e3d4-c402-4e59-8baa-08d77a20deda admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c947e76d727046b9ae485331a1e4211d'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:09:20.559153 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 961/1951] 10.222.0.46 () {58 vars in 1224 bytes} [Mon Oct 10 12:09:20 2022] DELETE /identity/v3/users/c947e76d727046b9ae485331a1e4211d => generated 0 bytes in 522 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 10 12:09:20.565180 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b01521-fe16-49e6-84ed-17c7bd695630 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:09:20.565985 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b01521-fe16-49e6-84ed-17c7bd695630 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:09:20.567189 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b01521-fe16-49e6-84ed-17c7bd695630 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:09:20.602916 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-94b01521-fe16-49e6-84ed-17c7bd695630 None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:09:20.605195 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-94b01521-fe16-49e6-84ed-17c7bd695630 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:09:20.704661 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 991/1952] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:09:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:09:20.715378 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf01e3d4-c402-4e59-8baa-08d77a20deda admin admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:09:20.716853 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-bf01e3d4-c402-4e59-8baa-08d77a20deda admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:09:20.778600 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf01e3d4-c402-4e59-8baa-08d77a20deda admin admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:09:20.782456 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4982e654-ad80-4491-bf47-1ef4f99edd7f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58b1c7c85302421cba823ace4201861f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad3772f7b6a47c28b7ba37a9316a3c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:09:20.783126 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4982e654-ad80-4491-bf47-1ef4f99edd7f service neutron] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:09:20.783406 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4982e654-ad80-4491-bf47-1ef4f99edd7f service neutron] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:09:20.783679 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4982e654-ad80-4491-bf47-1ef4f99edd7f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:09:20.793719 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4982e654-ad80-4491-bf47-1ef4f99edd7f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:09:20.795214 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4982e654-ad80-4491-bf47-1ef4f99edd7f service neutron] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:09:20.806700 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 962/1953] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 10 12:09:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:09:21.286766 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b01521-fe16-49e6-84ed-17c7bd695630 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:09:21.289299 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-94b01521-fe16-49e6-84ed-17c7bd695630 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:09:21.303669 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b01521-fe16-49e6-84ed-17c7bd695630 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:09:21.312212 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56bb4f7-1df2-420e-b49c-3b68e503e9b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:09:21.313746 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56bb4f7-1df2-420e-b49c-3b68e503e9b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:09:21.314451 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56bb4f7-1df2-420e-b49c-3b68e503e9b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:09:21.315318 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56bb4f7-1df2-420e-b49c-3b68e503e9b5 admin admin] PATH_INFO: `/v3/projects/3386a2e61eca426983604c98af697463` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:09:21.318069 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56bb4f7-1df2-420e-b49c-3b68e503e9b5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3386a2e61eca426983604c98af697463)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:09:21.323101 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56bb4f7-1df2-420e-b49c-3b68e503e9b5 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:09:21.489252 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-c56bb4f7-1df2-420e-b49c-3b68e503e9b5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3386a2e61eca426983604c98af697463', 'request_id': 'req-c56bb4f7-1df2-420e-b49c-3b68e503e9b5'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:09:21.491767 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:09:21.498467 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-c56bb4f7-1df2-420e-b49c-3b68e503e9b5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3386a2e61eca426983604c98af697463', 'request_id': 'req-c56bb4f7-1df2-420e-b49c-3b68e503e9b5'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:09:21.505158 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-c56bb4f7-1df2-420e-b49c-3b68e503e9b5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3386a2e61eca426983604c98af697463', 'request_id': 'req-c56bb4f7-1df2-420e-b49c-3b68e503e9b5'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:09:21.657823 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-c56bb4f7-1df2-420e-b49c-3b68e503e9b5 admin admin] The token cache is being invalidate because project 3386a2e61eca426983604c98af697463 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81677) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 10 12:09:21.664227 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-c56bb4f7-1df2-420e-b49c-3b68e503e9b5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:09:21.664227 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-c56bb4f7-1df2-420e-b49c-3b68e503e9b5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:09:21.664227 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 992/1954] 10.222.0.46 () {58 vars in 1233 bytes} [Mon Oct 10 12:09:21 2022] DELETE /identity/v3/projects/3386a2e61eca426983604c98af697463 => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 10 12:09:21.666734 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bbc86c-4efa-436b-8ea2-ee69e22f9c4a None None] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:09:21.667225 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bbc86c-4efa-436b-8ea2-ee69e22f9c4a None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:09:21.667375 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bbc86c-4efa-436b-8ea2-ee69e22f9c4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:09:21.689535 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.auth.core [None req-b7bbc86c-4efa-436b-8ea2-ee69e22f9c4a None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:09:21.690714 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-b7bbc86c-4efa-436b-8ea2-ee69e22f9c4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:09:21.756286 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:09:21.770517 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 963/1955] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:09:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 10 12:09:21.778729 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba869b4-cb69-4050-ac7b-9adb57dec85d None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:09:21.779361 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba869b4-cb69-4050-ac7b-9adb57dec85d None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:09:21.779649 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba869b4-cb69-4050-ac7b-9adb57dec85d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:09:21.808993 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-dba869b4-cb69-4050-ac7b-9adb57dec85d None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:09:21.810285 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-dba869b4-cb69-4050-ac7b-9adb57dec85d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:09:21.819340 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 993/1956] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:09:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:09:21.825761 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7bbc86c-4efa-436b-8ea2-ee69e22f9c4a None None] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:09:21.841637 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7bbc86c-4efa-436b-8ea2-ee69e22f9c4a None None] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:09:21.849717 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad397ff8-8467-4a10-b0ca-7b3ff213b33a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:09:21.850992 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad397ff8-8467-4a10-b0ca-7b3ff213b33a admin admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:09:21.851613 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad397ff8-8467-4a10-b0ca-7b3ff213b33a admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:09:21.852444 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad397ff8-8467-4a10-b0ca-7b3ff213b33a admin admin] PATH_INFO: `/v3/domains` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:09:21.853972 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad397ff8-8467-4a10-b0ca-7b3ff213b33a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 12:09:21.854528 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad397ff8-8467-4a10-b0ca-7b3ff213b33a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:09:21.857787 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad397ff8-8467-4a10-b0ca-7b3ff213b33a admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:09:21.880587 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 964/1957] 10.222.0.46 () {58 vars in 1153 bytes} [Mon Oct 10 12:09:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 12:09:21.888802 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba869b4-cb69-4050-ac7b-9adb57dec85d None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:09:21.909752 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba869b4-cb69-4050-ac7b-9adb57dec85d None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:09:21.922959 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528f9275-cce9-4500-a28f-060bfbbb18b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:09:21.923588 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528f9275-cce9-4500-a28f-060bfbbb18b7 admin admin] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:09:21.923901 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528f9275-cce9-4500-a28f-060bfbbb18b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:09:21.924797 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528f9275-cce9-4500-a28f-060bfbbb18b7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:09:21.928280 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528f9275-cce9-4500-a28f-060bfbbb18b7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:09:21.933376 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528f9275-cce9-4500-a28f-060bfbbb18b7 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:09:22.092560 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 994/1958] 10.222.0.46 () {60 vars in 1153 bytes} [Mon Oct 10 12:09:21 2022] POST /identity/v3/projects => generated 373 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 12:09:22.097193 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad397ff8-8467-4a10-b0ca-7b3ff213b33a admin admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:09:22.107638 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad397ff8-8467-4a10-b0ca-7b3ff213b33a admin admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:09:22.173256 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0f667d-a146-410d-aefc-75843f0298f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:09:22.173886 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0f667d-a146-410d-aefc-75843f0298f5 admin admin] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:09:22.174168 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0f667d-a146-410d-aefc-75843f0298f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:09:22.174443 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0f667d-a146-410d-aefc-75843f0298f5 admin admin] PATH_INFO: `/v3/users` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:09:22.175272 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0f667d-a146-410d-aefc-75843f0298f5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:09:22.178526 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0f667d-a146-410d-aefc-75843f0298f5 admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:09:22.291456 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 965/1959] 10.222.0.46 () {60 vars in 1144 bytes} [Mon Oct 10 12:09:22 2022] POST /identity/v3/users => generated 287 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 10 12:09:22.296154 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528f9275-cce9-4500-a28f-060bfbbb18b7 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:09:22.308305 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528f9275-cce9-4500-a28f-060bfbbb18b7 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:09:22.313107 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6af232a-6389-4ed8-a5d6-487622159e71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:09:22.313740 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6af232a-6389-4ed8-a5d6-487622159e71 admin admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:09:22.314019 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6af232a-6389-4ed8-a5d6-487622159e71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:09:22.314301 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6af232a-6389-4ed8-a5d6-487622159e71 admin admin] PATH_INFO: `/v3/roles` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:09:22.314992 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6af232a-6389-4ed8-a5d6-487622159e71 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:09:22.317218 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6af232a-6389-4ed8-a5d6-487622159e71 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:09:22.327923 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 995/1960] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 10 12:09:22 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 10 12:09:22.332497 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0f667d-a146-410d-aefc-75843f0298f5 admin admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:09:22.342913 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0f667d-a146-410d-aefc-75843f0298f5 admin admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:09:22.347888 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4ffed6-dcde-4268-947e-2210c3ab32d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:09:22.348512 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4ffed6-dcde-4268-947e-2210c3ab32d2 admin admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:09:22.355289 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4ffed6-dcde-4268-947e-2210c3ab32d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:09:22.355289 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4ffed6-dcde-4268-947e-2210c3ab32d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:09:22.355289 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4ffed6-dcde-4268-947e-2210c3ab32d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:09:22.355289 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4ffed6-dcde-4268-947e-2210c3ab32d2 admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:09:22.363895 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 966/1961] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 10 12:09:22 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 10 12:09:22.368506 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6af232a-6389-4ed8-a5d6-487622159e71 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:09:22.381782 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6af232a-6389-4ed8-a5d6-487622159e71 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:09:22.387070 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1419d2-018b-435c-8eab-eebbc97cda7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:09:22.387804 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1419d2-018b-435c-8eab-eebbc97cda7c admin admin] REQUEST_METHOD: `PUT` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:09:22.388156 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1419d2-018b-435c-8eab-eebbc97cda7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:09:22.388515 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1419d2-018b-435c-8eab-eebbc97cda7c admin admin] PATH_INFO: `/v3/projects/542cd4bec9244d86853ce11782bf5ead/users/70b34bc322c04ee287da162a1fe02a61/roles/b2cb2a6e818e47338ce071e6fedfc673` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:09:22.406564 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1419d2-018b-435c-8eab-eebbc97cda7c admin admin] RBAC: Authorizing `identity:create_grant(project_id=542cd4bec9244d86853ce11782bf5ead, user_id=70b34bc322c04ee287da162a1fe02a61, role_id=b2cb2a6e818e47338ce071e6fedfc673)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:09:22.409334 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1419d2-018b-435c-8eab-eebbc97cda7c admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:09:22.536215 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 996/1962] 10.222.0.46 () {60 vars in 1483 bytes} [Mon Oct 10 12:09:22 2022] PUT /identity/v3/projects/542cd4bec9244d86853ce11782bf5ead/users/70b34bc322c04ee287da162a1fe02a61/roles/b2cb2a6e818e47338ce071e6fedfc673 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:09:22.547591 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91368211-cc53-401d-8b03-ba12f4aa4e1e None None] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:09:22.547859 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91368211-cc53-401d-8b03-ba12f4aa4e1e None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:09:22.548188 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91368211-cc53-401d-8b03-ba12f4aa4e1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:09:22.580003 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.auth.core [None req-91368211-cc53-401d-8b03-ba12f4aa4e1e None None] MFA Rules not processed for user `70b34bc322c04ee287da162a1fe02a61`. Rule list: `[]` (Enabled: `True`). {{(pid=81676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:09:22.581247 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-91368211-cc53-401d-8b03-ba12f4aa4e1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:09:22.658045 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 967/1963] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:09:22 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 10 12:09:22.670108 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1419d2-018b-435c-8eab-eebbc97cda7c admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:09:22.671926 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-bc1419d2-018b-435c-8eab-eebbc97cda7c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:09:22.735882 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1419d2-018b-435c-8eab-eebbc97cda7c admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:09:22.740273 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe2e5bb-6c9b-432d-a414-b043e4b497f4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3ae093470484b93b23dcfae1657df9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad3772f7b6a47c28b7ba37a9316a3c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:09:22.740897 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe2e5bb-6c9b-432d-a414-b043e4b497f4 service cinder] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:09:22.741180 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe2e5bb-6c9b-432d-a414-b043e4b497f4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:09:22.741504 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe2e5bb-6c9b-432d-a414-b043e4b497f4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:09:22.750485 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe2e5bb-6c9b-432d-a414-b043e4b497f4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:09:22.751950 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe2e5bb-6c9b-432d-a414-b043e4b497f4 service cinder] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:09:22.761775 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 997/1964] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 10 12:09:22 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:09:24.460451 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91368211-cc53-401d-8b03-ba12f4aa4e1e None None] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:09:24.468535 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-91368211-cc53-401d-8b03-ba12f4aa4e1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:09:24.569800 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:09:24.588635 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91368211-cc53-401d-8b03-ba12f4aa4e1e None None] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:09:24.593813 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34011c5d-3f48-45b3-b4b7-cad1f153d3aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:09:24.594635 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34011c5d-3f48-45b3-b4b7-cad1f153d3aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:09:24.595029 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34011c5d-3f48-45b3-b4b7-cad1f153d3aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:09:24.595376 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34011c5d-3f48-45b3-b4b7-cad1f153d3aa admin admin] PATH_INFO: `/v3/users/b2815f06069a4d7bba28ff8a2be4d5d8` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:09:24.596934 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34011c5d-3f48-45b3-b4b7-cad1f153d3aa admin admin] RBAC: Authorizing `identity:delete_user(user_id=b2815f06069a4d7bba28ff8a2be4d5d8)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:09:24.599742 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34011c5d-3f48-45b3-b4b7-cad1f153d3aa admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:09:24.932793 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-34011c5d-3f48-45b3-b4b7-cad1f153d3aa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b2815f06069a4d7bba28ff8a2be4d5d8'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:09:24.946498 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-34011c5d-3f48-45b3-b4b7-cad1f153d3aa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b2815f06069a4d7bba28ff8a2be4d5d8'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:09:24.947829 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-34011c5d-3f48-45b3-b4b7-cad1f153d3aa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b2815f06069a4d7bba28ff8a2be4d5d8'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:09:25.068756 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-34011c5d-3f48-45b3-b4b7-cad1f153d3aa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b2815f06069a4d7bba28ff8a2be4d5d8'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:09:25.093109 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-34011c5d-3f48-45b3-b4b7-cad1f153d3aa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b2815f06069a4d7bba28ff8a2be4d5d8'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:09:25.106509 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 968/1965] 10.222.0.46 () {58 vars in 1224 bytes} [Mon Oct 10 12:09:24 2022] DELETE /identity/v3/users/b2815f06069a4d7bba28ff8a2be4d5d8 => generated 0 bytes in 645 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:09:25.109768 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7082310d-d0cb-4b3d-a656-b84e1a030349 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:09:25.110606 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7082310d-d0cb-4b3d-a656-b84e1a030349 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:09:25.111022 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7082310d-d0cb-4b3d-a656-b84e1a030349 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:09:25.138570 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-7082310d-d0cb-4b3d-a656-b84e1a030349 None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:09:25.140484 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-7082310d-d0cb-4b3d-a656-b84e1a030349 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:09:25.233723 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 998/1966] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:09:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:09:25.247519 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34011c5d-3f48-45b3-b4b7-cad1f153d3aa admin admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:09:25.249591 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-34011c5d-3f48-45b3-b4b7-cad1f153d3aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:09:25.346970 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34011c5d-3f48-45b3-b4b7-cad1f153d3aa admin admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:09:25.356948 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4214292d-9b59-4d1d-bbaf-c256a466db41 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58b1c7c85302421cba823ace4201861f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad3772f7b6a47c28b7ba37a9316a3c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:09:25.357998 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4214292d-9b59-4d1d-bbaf-c256a466db41 service neutron] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:09:25.359034 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4214292d-9b59-4d1d-bbaf-c256a466db41 service neutron] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:09:25.359155 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4214292d-9b59-4d1d-bbaf-c256a466db41 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:09:25.381504 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4214292d-9b59-4d1d-bbaf-c256a466db41 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:09:25.384878 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4214292d-9b59-4d1d-bbaf-c256a466db41 service neutron] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:09:25.405659 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 969/1967] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 10 12:09:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:09:26.113627 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7082310d-d0cb-4b3d-a656-b84e1a030349 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:09:26.114893 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-7082310d-d0cb-4b3d-a656-b84e1a030349 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:09:26.127499 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7082310d-d0cb-4b3d-a656-b84e1a030349 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:09:26.133064 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d50e391-db37-4fd1-ac1b-74d80b6dc49d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:09:26.133728 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d50e391-db37-4fd1-ac1b-74d80b6dc49d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:09:26.134010 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d50e391-db37-4fd1-ac1b-74d80b6dc49d admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:09:26.134286 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d50e391-db37-4fd1-ac1b-74d80b6dc49d admin admin] PATH_INFO: `/v3/users/e77285922e2641969b8156205afeccff` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:09:26.135543 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d50e391-db37-4fd1-ac1b-74d80b6dc49d admin admin] RBAC: Authorizing `identity:delete_user(user_id=e77285922e2641969b8156205afeccff)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:09:26.137594 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d50e391-db37-4fd1-ac1b-74d80b6dc49d admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:09:26.156725 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=94,keystone:DELETE=3,keystone:INSERT=3 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:09:26.227529 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4214292d-9b59-4d1d-bbaf-c256a466db41 service neutron] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:09:26.229572 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-4214292d-9b59-4d1d-bbaf-c256a466db41 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:09:26.258631 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4214292d-9b59-4d1d-bbaf-c256a466db41 service neutron] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:09:26.266047 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbb3113-a7a5-4436-9ecb-7899dbbce20c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:09:26.267201 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbb3113-a7a5-4436-9ecb-7899dbbce20c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:09:26.267442 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbb3113-a7a5-4436-9ecb-7899dbbce20c admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:09:26.267872 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbb3113-a7a5-4436-9ecb-7899dbbce20c admin admin] PATH_INFO: `/v3/projects/ef94f5b81a1e45b8bb56085399ce0f4c` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:09:26.269270 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbbb3113-a7a5-4436-9ecb-7899dbbce20c admin admin] RBAC: Authorizing `identity:delete_project(project_id=ef94f5b81a1e45b8bb56085399ce0f4c)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:09:26.271899 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbbb3113-a7a5-4436-9ecb-7899dbbce20c admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:09:26.406054 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-5d50e391-db37-4fd1-ac1b-74d80b6dc49d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e77285922e2641969b8156205afeccff'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:09:26.416384 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-5d50e391-db37-4fd1-ac1b-74d80b6dc49d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e77285922e2641969b8156205afeccff'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:09:26.417184 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-5d50e391-db37-4fd1-ac1b-74d80b6dc49d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e77285922e2641969b8156205afeccff'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:09:26.447940 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-cbbb3113-a7a5-4436-9ecb-7899dbbce20c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ef94f5b81a1e45b8bb56085399ce0f4c', 'request_id': 'req-cbbb3113-a7a5-4436-9ecb-7899dbbce20c'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:09:26.454248 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-cbbb3113-a7a5-4436-9ecb-7899dbbce20c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ef94f5b81a1e45b8bb56085399ce0f4c', 'request_id': 'req-cbbb3113-a7a5-4436-9ecb-7899dbbce20c'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:09:26.468439 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-cbbb3113-a7a5-4436-9ecb-7899dbbce20c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ef94f5b81a1e45b8bb56085399ce0f4c', 'request_id': 'req-cbbb3113-a7a5-4436-9ecb-7899dbbce20c'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:09:26.476100 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:09:26.546961 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-5d50e391-db37-4fd1-ac1b-74d80b6dc49d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e77285922e2641969b8156205afeccff'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:09:26.564662 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-5d50e391-db37-4fd1-ac1b-74d80b6dc49d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e77285922e2641969b8156205afeccff'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:09:26.573234 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 999/1968] 10.222.0.46 () {58 vars in 1224 bytes} [Mon Oct 10 12:09:26 2022] DELETE /identity/v3/users/e77285922e2641969b8156205afeccff => generated 0 bytes in 461 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 10 12:09:26.577615 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7f389d-f624-4a3f-9243-021375ad5ce7 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:09:26.579799 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7f389d-f624-4a3f-9243-021375ad5ce7 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:09:26.579799 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7f389d-f624-4a3f-9243-021375ad5ce7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:09:26.597390 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-1e7f389d-f624-4a3f-9243-021375ad5ce7 None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:09:26.599145 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-1e7f389d-f624-4a3f-9243-021375ad5ce7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:09:26.647185 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-cbbb3113-a7a5-4436-9ecb-7899dbbce20c admin admin] The token cache is being invalidate because project ef94f5b81a1e45b8bb56085399ce0f4c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81676) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 10 12:09:26.647480 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-cbbb3113-a7a5-4436-9ecb-7899dbbce20c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:09:26.648193 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-cbbb3113-a7a5-4436-9ecb-7899dbbce20c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:09:26.649756 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 970/1969] 10.222.0.46 () {58 vars in 1233 bytes} [Mon Oct 10 12:09:26 2022] DELETE /identity/v3/projects/ef94f5b81a1e45b8bb56085399ce0f4c => generated 0 bytes in 427 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 10 12:09:26.655417 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbb3113-a7a5-4436-9ecb-7899dbbce20c admin admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:09:26.660701 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-cbbb3113-a7a5-4436-9ecb-7899dbbce20c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:09:26.662610 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1000/1970] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:09:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 10 12:09:26.672974 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7f389d-f624-4a3f-9243-021375ad5ce7 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:09:26.674496 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-1e7f389d-f624-4a3f-9243-021375ad5ce7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:09:26.734290 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbb3113-a7a5-4436-9ecb-7899dbbce20c admin admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:09:26.736538 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7f389d-f624-4a3f-9243-021375ad5ce7 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:09:26.739483 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf8866b-a707-4b5a-bb57-261b446f7cab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:09:26.740262 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9cad4a-c70e-457e-90d4-6c297a875a77 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58b1c7c85302421cba823ace4201861f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad3772f7b6a47c28b7ba37a9316a3c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:09:26.740438 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf8866b-a707-4b5a-bb57-261b446f7cab admin admin] REQUEST_METHOD: `DELETE` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:09:26.740614 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf8866b-a707-4b5a-bb57-261b446f7cab admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:09:26.740876 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9cad4a-c70e-457e-90d4-6c297a875a77 service neutron] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:09:26.741007 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf8866b-a707-4b5a-bb57-261b446f7cab admin admin] PATH_INFO: `/v3/users/1f198cb8e6714a619d7cbd985d3cace1` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:09:26.741157 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9cad4a-c70e-457e-90d4-6c297a875a77 service neutron] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:09:26.741542 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9cad4a-c70e-457e-90d4-6c297a875a77 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:09:26.743484 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf8866b-a707-4b5a-bb57-261b446f7cab admin admin] RBAC: Authorizing `identity:delete_user(user_id=1f198cb8e6714a619d7cbd985d3cace1)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:09:26.753774 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf8866b-a707-4b5a-bb57-261b446f7cab admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:09:26.774914 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9cad4a-c70e-457e-90d4-6c297a875a77 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:09:26.776757 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9cad4a-c70e-457e-90d4-6c297a875a77 service neutron] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:09:26.791578 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1001/1971] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 10 12:09:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 10 12:09:27.098420 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-cdf8866b-a707-4b5a-bb57-261b446f7cab admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1f198cb8e6714a619d7cbd985d3cace1'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:09:27.112913 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-cdf8866b-a707-4b5a-bb57-261b446f7cab admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1f198cb8e6714a619d7cbd985d3cace1'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:09:27.113848 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-cdf8866b-a707-4b5a-bb57-261b446f7cab admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1f198cb8e6714a619d7cbd985d3cace1'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:09:27.240207 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-cdf8866b-a707-4b5a-bb57-261b446f7cab admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1f198cb8e6714a619d7cbd985d3cace1'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:09:27.257187 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-cdf8866b-a707-4b5a-bb57-261b446f7cab admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1f198cb8e6714a619d7cbd985d3cace1'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:09:27.263405 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 971/1972] 10.222.0.46 () {58 vars in 1224 bytes} [Mon Oct 10 12:09:26 2022] DELETE /identity/v3/users/1f198cb8e6714a619d7cbd985d3cace1 => generated 0 bytes in 609 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 10 12:09:27.440440 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9cad4a-c70e-457e-90d4-6c297a875a77 service neutron] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:09:27.443590 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-4b9cad4a-c70e-457e-90d4-6c297a875a77 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:09:27.481206 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9cad4a-c70e-457e-90d4-6c297a875a77 service neutron] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:09:27.542435 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:09:27.571362 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33018f0b-c6ae-4093-84fa-6eb5bc09063f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:09:27.572878 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33018f0b-c6ae-4093-84fa-6eb5bc09063f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:09:27.573187 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33018f0b-c6ae-4093-84fa-6eb5bc09063f admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:09:27.573575 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33018f0b-c6ae-4093-84fa-6eb5bc09063f admin admin] PATH_INFO: `/v3/projects/771f46b5f15044a9a19d1f8f047adfb8` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:09:27.574995 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33018f0b-c6ae-4093-84fa-6eb5bc09063f admin admin] RBAC: Authorizing `identity:delete_project(project_id=771f46b5f15044a9a19d1f8f047adfb8)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:09:27.577418 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33018f0b-c6ae-4093-84fa-6eb5bc09063f admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:09:27.761369 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-33018f0b-c6ae-4093-84fa-6eb5bc09063f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '771f46b5f15044a9a19d1f8f047adfb8', 'request_id': 'req-33018f0b-c6ae-4093-84fa-6eb5bc09063f'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:09:27.770689 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-33018f0b-c6ae-4093-84fa-6eb5bc09063f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '771f46b5f15044a9a19d1f8f047adfb8', 'request_id': 'req-33018f0b-c6ae-4093-84fa-6eb5bc09063f'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:09:27.779690 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-33018f0b-c6ae-4093-84fa-6eb5bc09063f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '771f46b5f15044a9a19d1f8f047adfb8', 'request_id': 'req-33018f0b-c6ae-4093-84fa-6eb5bc09063f'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:09:27.862909 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-33018f0b-c6ae-4093-84fa-6eb5bc09063f admin admin] The token cache is being invalidate because project 771f46b5f15044a9a19d1f8f047adfb8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81677) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 10 12:09:27.864099 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-33018f0b-c6ae-4093-84fa-6eb5bc09063f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:09:27.865902 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-33018f0b-c6ae-4093-84fa-6eb5bc09063f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:09:27.869814 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1002/1973] 10.222.0.46 () {58 vars in 1233 bytes} [Mon Oct 10 12:09:27 2022] DELETE /identity/v3/projects/771f46b5f15044a9a19d1f8f047adfb8 => generated 0 bytes in 430 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:09:27.874567 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf8866b-a707-4b5a-bb57-261b446f7cab admin admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:09:27.878391 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-cdf8866b-a707-4b5a-bb57-261b446f7cab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:09:27.899320 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf8866b-a707-4b5a-bb57-261b446f7cab admin admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:09:27.956706 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33018f0b-c6ae-4093-84fa-6eb5bc09063f admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:09:27.958447 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-33018f0b-c6ae-4093-84fa-6eb5bc09063f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:09:28.006450 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f055d5f-982c-4ca9-8734-5b80dac84ee6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:09:28.007869 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f055d5f-982c-4ca9-8734-5b80dac84ee6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:09:28.008264 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f055d5f-982c-4ca9-8734-5b80dac84ee6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:09:28.008727 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f055d5f-982c-4ca9-8734-5b80dac84ee6 admin admin] PATH_INFO: `/v3/users/96d72ee58d6743929e2f009e32010804` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:09:28.012016 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f055d5f-982c-4ca9-8734-5b80dac84ee6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=96d72ee58d6743929e2f009e32010804)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:09:28.015132 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f055d5f-982c-4ca9-8734-5b80dac84ee6 admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:09:28.041604 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:09:28.094863 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33018f0b-c6ae-4093-84fa-6eb5bc09063f admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:09:28.111365 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e79b72e-879b-432e-883e-84c4657bba41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:09:28.112650 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e79b72e-879b-432e-883e-84c4657bba41 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:09:28.112760 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e79b72e-879b-432e-883e-84c4657bba41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:09:28.113013 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e79b72e-879b-432e-883e-84c4657bba41 admin admin] PATH_INFO: `/v3/projects/eab252b092424f8b868913277c9b66f9` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:09:28.122308 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e79b72e-879b-432e-883e-84c4657bba41 admin admin] RBAC: Authorizing `identity:delete_project(project_id=eab252b092424f8b868913277c9b66f9)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:09:28.122308 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e79b72e-879b-432e-883e-84c4657bba41 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:09:28.244611 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-6f055d5f-982c-4ca9-8734-5b80dac84ee6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '96d72ee58d6743929e2f009e32010804'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:09:28.272501 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-6f055d5f-982c-4ca9-8734-5b80dac84ee6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '96d72ee58d6743929e2f009e32010804'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:09:28.279615 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-6f055d5f-982c-4ca9-8734-5b80dac84ee6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '96d72ee58d6743929e2f009e32010804'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:09:28.295077 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-8e79b72e-879b-432e-883e-84c4657bba41 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eab252b092424f8b868913277c9b66f9', 'request_id': 'req-8e79b72e-879b-432e-883e-84c4657bba41'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:09:28.309894 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-8e79b72e-879b-432e-883e-84c4657bba41 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eab252b092424f8b868913277c9b66f9', 'request_id': 'req-8e79b72e-879b-432e-883e-84c4657bba41'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:09:28.318442 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-8e79b72e-879b-432e-883e-84c4657bba41 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eab252b092424f8b868913277c9b66f9', 'request_id': 'req-8e79b72e-879b-432e-883e-84c4657bba41'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:09:28.435676 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-6f055d5f-982c-4ca9-8734-5b80dac84ee6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '96d72ee58d6743929e2f009e32010804'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:09:28.449786 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-6f055d5f-982c-4ca9-8734-5b80dac84ee6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '96d72ee58d6743929e2f009e32010804'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:09:28.461099 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 972/1974] 10.222.0.46 () {58 vars in 1224 bytes} [Mon Oct 10 12:09:27 2022] DELETE /identity/v3/users/96d72ee58d6743929e2f009e32010804 => generated 0 bytes in 589 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:09:28.466657 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f055d5f-982c-4ca9-8734-5b80dac84ee6 admin admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:09:28.469227 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-6f055d5f-982c-4ca9-8734-5b80dac84ee6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:09:28.552214 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f055d5f-982c-4ca9-8734-5b80dac84ee6 admin admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:09:28.558764 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eda9c1a-4e49-4802-8d11-9a53894e1e76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:09:28.559796 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eda9c1a-4e49-4802-8d11-9a53894e1e76 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:09:28.560173 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eda9c1a-4e49-4802-8d11-9a53894e1e76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:09:28.560537 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eda9c1a-4e49-4802-8d11-9a53894e1e76 admin admin] PATH_INFO: `/v3/users/6bc73e7bcf19443a940b6969e1be8e47` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:09:28.562024 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eda9c1a-4e49-4802-8d11-9a53894e1e76 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6bc73e7bcf19443a940b6969e1be8e47)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:09:28.564527 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eda9c1a-4e49-4802-8d11-9a53894e1e76 admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:09:28.583397 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-8e79b72e-879b-432e-883e-84c4657bba41 admin admin] The token cache is being invalidate because project eab252b092424f8b868913277c9b66f9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81677) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 10 12:09:28.584298 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-8e79b72e-879b-432e-883e-84c4657bba41 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:09:28.585232 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-8e79b72e-879b-432e-883e-84c4657bba41 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:09:28.587652 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1003/1975] 10.222.0.46 () {58 vars in 1233 bytes} [Mon Oct 10 12:09:27 2022] DELETE /identity/v3/projects/eab252b092424f8b868913277c9b66f9 => generated 0 bytes in 633 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:09:28.596500 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ee4dd2-3394-42a3-a369-11c9b311c21d None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:09:28.597066 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ee4dd2-3394-42a3-a369-11c9b311c21d None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:09:28.597665 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ee4dd2-3394-42a3-a369-11c9b311c21d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:09:28.622091 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-40ee4dd2-3394-42a3-a369-11c9b311c21d None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:09:28.623355 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-40ee4dd2-3394-42a3-a369-11c9b311c21d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:09:28.632336 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1004/1976] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:09:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:09:28.642417 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0b59e0-a26e-405a-9f3e-b3ce66b40b52 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:09:28.642888 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0b59e0-a26e-405a-9f3e-b3ce66b40b52 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:09:28.643175 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0b59e0-a26e-405a-9f3e-b3ce66b40b52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:09:28.664239 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-de0b59e0-a26e-405a-9f3e-b3ce66b40b52 None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:09:28.665479 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-de0b59e0-a26e-405a-9f3e-b3ce66b40b52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:09:28.674129 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1005/1977] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:09:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 10 12:09:28.678390 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0b59e0-a26e-405a-9f3e-b3ce66b40b52 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:09:28.688018 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0b59e0-a26e-405a-9f3e-b3ce66b40b52 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:09:28.693385 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9031039-adb4-4e76-a692-520c1a53cb2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:09:28.694038 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9031039-adb4-4e76-a692-520c1a53cb2b admin admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:09:28.694361 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9031039-adb4-4e76-a692-520c1a53cb2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:09:28.694635 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9031039-adb4-4e76-a692-520c1a53cb2b admin admin] PATH_INFO: `/v3/domains` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:09:28.695326 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9031039-adb4-4e76-a692-520c1a53cb2b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 12:09:28.695631 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9031039-adb4-4e76-a692-520c1a53cb2b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:09:28.697264 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9031039-adb4-4e76-a692-520c1a53cb2b admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:09:28.710129 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1006/1978] 10.222.0.46 () {58 vars in 1153 bytes} [Mon Oct 10 12:09:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 12:09:28.715938 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9031039-adb4-4e76-a692-520c1a53cb2b admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:09:28.731367 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9031039-adb4-4e76-a692-520c1a53cb2b admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:09:28.740136 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb1cb03-e293-43e9-94d9-40e6622217dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:09:28.741384 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb1cb03-e293-43e9-94d9-40e6622217dc admin admin] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:09:28.741918 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb1cb03-e293-43e9-94d9-40e6622217dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:09:28.742520 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb1cb03-e293-43e9-94d9-40e6622217dc admin admin] PATH_INFO: `/v3/projects` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:09:28.743693 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb1cb03-e293-43e9-94d9-40e6622217dc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:09:28.747598 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb1cb03-e293-43e9-94d9-40e6622217dc admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:09:28.818465 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-7eda9c1a-4e49-4802-8d11-9a53894e1e76 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6bc73e7bcf19443a940b6969e1be8e47'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:09:28.826991 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-7eda9c1a-4e49-4802-8d11-9a53894e1e76 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6bc73e7bcf19443a940b6969e1be8e47'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:09:28.827776 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-7eda9c1a-4e49-4802-8d11-9a53894e1e76 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6bc73e7bcf19443a940b6969e1be8e47'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:09:28.835106 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:09:28.843861 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1007/1979] 10.222.0.46 () {60 vars in 1153 bytes} [Mon Oct 10 12:09:28 2022] POST /identity/v3/projects => generated 375 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 10 12:09:28.848248 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb1cb03-e293-43e9-94d9-40e6622217dc admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:09:28.850387 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-1fb1cb03-e293-43e9-94d9-40e6622217dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:09:28.920923 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb1cb03-e293-43e9-94d9-40e6622217dc admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:09:28.926329 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b4b30f-9b69-41fe-8068-f7c8c9cfa914 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:09:28.927121 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b4b30f-9b69-41fe-8068-f7c8c9cfa914 admin admin] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:09:28.927401 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b4b30f-9b69-41fe-8068-f7c8c9cfa914 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:09:28.927675 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b4b30f-9b69-41fe-8068-f7c8c9cfa914 admin admin] PATH_INFO: `/v3/users` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:09:28.928325 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b4b30f-9b69-41fe-8068-f7c8c9cfa914 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:09:28.930321 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b4b30f-9b69-41fe-8068-f7c8c9cfa914 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:09:29.058067 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-7eda9c1a-4e49-4802-8d11-9a53894e1e76 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6bc73e7bcf19443a940b6969e1be8e47'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:09:29.071746 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-7eda9c1a-4e49-4802-8d11-9a53894e1e76 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6bc73e7bcf19443a940b6969e1be8e47'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:09:29.077924 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 973/1980] 10.222.0.46 () {58 vars in 1224 bytes} [Mon Oct 10 12:09:28 2022] DELETE /identity/v3/users/6bc73e7bcf19443a940b6969e1be8e47 => generated 0 bytes in 616 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:09:29.095645 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eda9c1a-4e49-4802-8d11-9a53894e1e76 admin admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:09:29.096184 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1008/1981] 10.222.0.46 () {60 vars in 1144 bytes} [Mon Oct 10 12:09:28 2022] POST /identity/v3/users => generated 288 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 10 12:09:29.102009 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b4b30f-9b69-41fe-8068-f7c8c9cfa914 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:09:29.103648 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-7eda9c1a-4e49-4802-8d11-9a53894e1e76 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:09:29.115712 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b4b30f-9b69-41fe-8068-f7c8c9cfa914 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:09:29.196841 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eda9c1a-4e49-4802-8d11-9a53894e1e76 admin admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:09:29.201606 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fcd6189-b354-41ed-9813-5c4c7df3b38e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:09:29.202335 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcd6189-b354-41ed-9813-5c4c7df3b38e admin admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:09:29.202753 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcd6189-b354-41ed-9813-5c4c7df3b38e admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:09:29.203038 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcd6189-b354-41ed-9813-5c4c7df3b38e admin admin] PATH_INFO: `/v3/roles` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:09:29.203801 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fcd6189-b354-41ed-9813-5c4c7df3b38e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:09:29.203909 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317e8bc4-ed21-46b0-8383-1f5c3c82a67e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:09:29.204554 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317e8bc4-ed21-46b0-8383-1f5c3c82a67e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:09:29.204842 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317e8bc4-ed21-46b0-8383-1f5c3c82a67e admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:09:29.205152 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317e8bc4-ed21-46b0-8383-1f5c3c82a67e admin admin] PATH_INFO: `/v3/projects/ed1233dffa344afe926708081a99651d` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:09:29.206385 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317e8bc4-ed21-46b0-8383-1f5c3c82a67e admin admin] RBAC: Authorizing `identity:delete_project(project_id=ed1233dffa344afe926708081a99651d)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:09:29.206690 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fcd6189-b354-41ed-9813-5c4c7df3b38e admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:09:29.211097 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317e8bc4-ed21-46b0-8383-1f5c3c82a67e admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:09:29.224736 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1009/1982] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 10 12:09:29 2022] GET /identity/v3/roles => generated 1420 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 12:09:29.226635 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fcd6189-b354-41ed-9813-5c4c7df3b38e admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:09:29.239529 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fcd6189-b354-41ed-9813-5c4c7df3b38e admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:09:29.245341 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf4cdd1-25ff-4203-a388-1147902011a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:09:29.247830 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf4cdd1-25ff-4203-a388-1147902011a5 admin admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:09:29.247830 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf4cdd1-25ff-4203-a388-1147902011a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:09:29.247830 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf4cdd1-25ff-4203-a388-1147902011a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:09:29.247830 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf4cdd1-25ff-4203-a388-1147902011a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:09:29.249763 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf4cdd1-25ff-4203-a388-1147902011a5 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:09:29.266568 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1010/1983] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 10 12:09:29 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 12:09:29.266933 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf4cdd1-25ff-4203-a388-1147902011a5 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:09:29.280470 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf4cdd1-25ff-4203-a388-1147902011a5 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:09:29.288658 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0606e5af-0493-4596-a0aa-c168b1972204 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:09:29.289514 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0606e5af-0493-4596-a0aa-c168b1972204 admin admin] REQUEST_METHOD: `PUT` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:09:29.289789 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0606e5af-0493-4596-a0aa-c168b1972204 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:09:29.290096 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0606e5af-0493-4596-a0aa-c168b1972204 admin admin] PATH_INFO: `/v3/projects/8d6a8f944eb94b84bd67f0926ac637f8/users/03d3aaf04d9d436b97618e4419c6366b/roles/b2cb2a6e818e47338ce071e6fedfc673` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:09:29.306537 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0606e5af-0493-4596-a0aa-c168b1972204 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8d6a8f944eb94b84bd67f0926ac637f8, user_id=03d3aaf04d9d436b97618e4419c6366b, role_id=b2cb2a6e818e47338ce071e6fedfc673)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:09:29.317586 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0606e5af-0493-4596-a0aa-c168b1972204 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:09:29.354088 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:DELETE=19,keystone:SELECT=79,keystone:INSERT=2 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:09:29.429206 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1011/1984] 10.222.0.46 () {60 vars in 1483 bytes} [Mon Oct 10 12:09:29 2022] PUT /identity/v3/projects/8d6a8f944eb94b84bd67f0926ac637f8/users/03d3aaf04d9d436b97618e4419c6366b/roles/b2cb2a6e818e47338ce071e6fedfc673 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 10 12:09:29.430243 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0606e5af-0493-4596-a0aa-c168b1972204 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:09:29.431931 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-0606e5af-0493-4596-a0aa-c168b1972204 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:09:29.448275 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-317e8bc4-ed21-46b0-8383-1f5c3c82a67e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ed1233dffa344afe926708081a99651d', 'request_id': 'req-317e8bc4-ed21-46b0-8383-1f5c3c82a67e'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:09:29.468996 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-317e8bc4-ed21-46b0-8383-1f5c3c82a67e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ed1233dffa344afe926708081a99651d', 'request_id': 'req-317e8bc4-ed21-46b0-8383-1f5c3c82a67e'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:09:29.485433 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-317e8bc4-ed21-46b0-8383-1f5c3c82a67e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ed1233dffa344afe926708081a99651d', 'request_id': 'req-317e8bc4-ed21-46b0-8383-1f5c3c82a67e'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:09:29.526090 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0606e5af-0493-4596-a0aa-c168b1972204 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:09:29.593849 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e106df7f-9fe3-4276-96a9-0ccce8943766 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:09:29.594809 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e106df7f-9fe3-4276-96a9-0ccce8943766 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:09:29.594905 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e106df7f-9fe3-4276-96a9-0ccce8943766 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:09:29.595141 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e106df7f-9fe3-4276-96a9-0ccce8943766 admin admin] PATH_INFO: `/v3/projects/f87226283dcb4d6fb203cc5e06cf13b2` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:09:29.596356 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e106df7f-9fe3-4276-96a9-0ccce8943766 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f87226283dcb4d6fb203cc5e06cf13b2)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:09:29.599108 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e106df7f-9fe3-4276-96a9-0ccce8943766 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:09:29.631353 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-317e8bc4-ed21-46b0-8383-1f5c3c82a67e admin admin] The token cache is being invalidate because project ed1233dffa344afe926708081a99651d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81676) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 10 12:09:29.632511 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-317e8bc4-ed21-46b0-8383-1f5c3c82a67e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:09:29.633309 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-317e8bc4-ed21-46b0-8383-1f5c3c82a67e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:09:29.637437 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 974/1985] 10.222.0.46 () {58 vars in 1233 bytes} [Mon Oct 10 12:09:29 2022] DELETE /identity/v3/projects/ed1233dffa344afe926708081a99651d => generated 0 bytes in 541 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:09:29.639153 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf1c59b-9af7-497d-a563-95541ac26724 None None] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:09:29.639795 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf1c59b-9af7-497d-a563-95541ac26724 None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:09:29.640126 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf1c59b-9af7-497d-a563-95541ac26724 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:09:29.672200 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.auth.core [None req-cdf1c59b-9af7-497d-a563-95541ac26724 None None] MFA Rules not processed for user `03d3aaf04d9d436b97618e4419c6366b`. Rule list: `[]` (Enabled: `True`). {{(pid=81676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:09:29.673486 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-cdf1c59b-9af7-497d-a563-95541ac26724 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:09:29.753489 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 975/1986] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:09:29 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:09:29.755415 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe7d298-193c-475a-ac65-151a3200325a None None] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:09:29.756080 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe7d298-193c-475a-ac65-151a3200325a None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:09:29.756621 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe7d298-193c-475a-ac65-151a3200325a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:09:29.768157 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-e106df7f-9fe3-4276-96a9-0ccce8943766 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f87226283dcb4d6fb203cc5e06cf13b2', 'request_id': 'req-e106df7f-9fe3-4276-96a9-0ccce8943766'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:09:29.777565 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-e106df7f-9fe3-4276-96a9-0ccce8943766 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f87226283dcb4d6fb203cc5e06cf13b2', 'request_id': 'req-e106df7f-9fe3-4276-96a9-0ccce8943766'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:09:29.778936 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.auth.core [None req-5fe7d298-193c-475a-ac65-151a3200325a None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:09:29.780191 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-5fe7d298-193c-475a-ac65-151a3200325a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:09:29.786639 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-e106df7f-9fe3-4276-96a9-0ccce8943766 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f87226283dcb4d6fb203cc5e06cf13b2', 'request_id': 'req-e106df7f-9fe3-4276-96a9-0ccce8943766'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:09:29.852688 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 976/1987] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:09:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:09:29.853823 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe7d298-193c-475a-ac65-151a3200325a None None] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:09:29.865774 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-5fe7d298-193c-475a-ac65-151a3200325a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:09:29.878919 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe7d298-193c-475a-ac65-151a3200325a None None] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:09:29.886103 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8122610f-a8c0-4438-a4ae-fb1b3ba35acc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:09:29.887188 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8122610f-a8c0-4438-a4ae-fb1b3ba35acc admin admin] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:09:29.887684 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8122610f-a8c0-4438-a4ae-fb1b3ba35acc admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:09:29.888179 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8122610f-a8c0-4438-a4ae-fb1b3ba35acc admin admin] PATH_INFO: `/v3/projects` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:09:29.889453 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8122610f-a8c0-4438-a4ae-fb1b3ba35acc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:09:29.893378 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8122610f-a8c0-4438-a4ae-fb1b3ba35acc admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:09:30.013585 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-e106df7f-9fe3-4276-96a9-0ccce8943766 admin admin] The token cache is being invalidate because project f87226283dcb4d6fb203cc5e06cf13b2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81677) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 10 12:09:30.014043 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-e106df7f-9fe3-4276-96a9-0ccce8943766 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:09:30.014880 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-e106df7f-9fe3-4276-96a9-0ccce8943766 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:09:30.016759 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1012/1988] 10.222.0.46 () {58 vars in 1233 bytes} [Mon Oct 10 12:09:29 2022] DELETE /identity/v3/projects/f87226283dcb4d6fb203cc5e06cf13b2 => generated 0 bytes in 587 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:09:30.018486 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c17a8f1-82b2-4fac-bb19-7921bc59cb8e None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:09:30.018967 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c17a8f1-82b2-4fac-bb19-7921bc59cb8e None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:09:30.019928 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c17a8f1-82b2-4fac-bb19-7921bc59cb8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:09:30.041837 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:09:30.044767 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-9c17a8f1-82b2-4fac-bb19-7921bc59cb8e None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:09:30.046781 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-9c17a8f1-82b2-4fac-bb19-7921bc59cb8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:09:30.051666 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 977/1989] 10.222.0.46 () {60 vars in 1153 bytes} [Mon Oct 10 12:09:29 2022] POST /identity/v3/projects => generated 375 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 12:09:30.053884 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b251b35e-6931-4121-ad92-a270df0dc385 None None] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:09:30.054510 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b251b35e-6931-4121-ad92-a270df0dc385 None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:09:30.055298 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b251b35e-6931-4121-ad92-a270df0dc385 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:09:30.080836 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.auth.core [None req-b251b35e-6931-4121-ad92-a270df0dc385 None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:09:30.083096 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-b251b35e-6931-4121-ad92-a270df0dc385 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:09:30.155976 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1013/1990] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:09:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:09:30.158006 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c17a8f1-82b2-4fac-bb19-7921bc59cb8e None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:09:30.162593 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-9c17a8f1-82b2-4fac-bb19-7921bc59cb8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:09:30.180336 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 978/1991] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:09:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:09:30.180336 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b251b35e-6931-4121-ad92-a270df0dc385 None None] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:09:30.195109 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b251b35e-6931-4121-ad92-a270df0dc385 None None] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:09:30.195803 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c17a8f1-82b2-4fac-bb19-7921bc59cb8e None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:09:30.201930 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307755df-f4ae-474f-9b0f-911ab18ad736 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:09:30.202731 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307755df-f4ae-474f-9b0f-911ab18ad736 admin admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:09:30.203009 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307755df-f4ae-474f-9b0f-911ab18ad736 admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:09:30.203328 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307755df-f4ae-474f-9b0f-911ab18ad736 admin admin] PATH_INFO: `/v3/domains` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:09:30.203925 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d60f78f-514c-4345-a028-41ed76469a2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:09:30.204219 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307755df-f4ae-474f-9b0f-911ab18ad736 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 12:09:30.204485 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307755df-f4ae-474f-9b0f-911ab18ad736 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:09:30.204900 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d60f78f-514c-4345-a028-41ed76469a2b admin admin] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:09:30.205352 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d60f78f-514c-4345-a028-41ed76469a2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:09:30.205788 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d60f78f-514c-4345-a028-41ed76469a2b admin admin] PATH_INFO: `/v3/users` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:09:30.206611 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307755df-f4ae-474f-9b0f-911ab18ad736 admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:09:30.207454 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d60f78f-514c-4345-a028-41ed76469a2b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:09:30.210809 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d60f78f-514c-4345-a028-41ed76469a2b admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:09:30.223219 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 979/1992] 10.222.0.46 () {58 vars in 1153 bytes} [Mon Oct 10 12:09:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 12:09:30.224555 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9881d65-4c0b-49e6-bb9a-502dd73c7969 None None] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:09:30.224845 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9881d65-4c0b-49e6-bb9a-502dd73c7969 None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:09:30.225121 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9881d65-4c0b-49e6-bb9a-502dd73c7969 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:09:30.247558 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.auth.core [None req-e9881d65-4c0b-49e6-bb9a-502dd73c7969 None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:09:30.248947 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-e9881d65-4c0b-49e6-bb9a-502dd73c7969 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:09:30.259011 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 980/1993] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:09:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:09:30.260096 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9881d65-4c0b-49e6-bb9a-502dd73c7969 None None] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:09:30.272794 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9881d65-4c0b-49e6-bb9a-502dd73c7969 None None] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:09:30.278309 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa67e64-3174-422a-857a-593e349bdbf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:09:30.278867 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa67e64-3174-422a-857a-593e349bdbf8 admin admin] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:09:30.279155 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa67e64-3174-422a-857a-593e349bdbf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:09:30.279456 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa67e64-3174-422a-857a-593e349bdbf8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:09:30.280104 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa67e64-3174-422a-857a-593e349bdbf8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:09:30.282195 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa67e64-3174-422a-857a-593e349bdbf8 admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:09:30.311093 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1014/1994] 10.222.0.46 () {60 vars in 1144 bytes} [Mon Oct 10 12:09:30 2022] POST /identity/v3/users => generated 288 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 12:09:30.312234 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d60f78f-514c-4345-a028-41ed76469a2b admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:09:30.324518 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d60f78f-514c-4345-a028-41ed76469a2b admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:09:30.329594 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ac3c13-c7f9-4411-ad91-c605fbfcef19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:09:30.330245 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ac3c13-c7f9-4411-ad91-c605fbfcef19 admin admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:09:30.330702 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ac3c13-c7f9-4411-ad91-c605fbfcef19 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:09:30.330852 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ac3c13-c7f9-4411-ad91-c605fbfcef19 admin admin] PATH_INFO: `/v3/domains` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:09:30.331524 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ac3c13-c7f9-4411-ad91-c605fbfcef19 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 12:09:30.332972 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ac3c13-c7f9-4411-ad91-c605fbfcef19 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:09:30.334045 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ac3c13-c7f9-4411-ad91-c605fbfcef19 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:09:30.346931 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1015/1995] 10.222.0.46 () {58 vars in 1153 bytes} [Mon Oct 10 12:09:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 12:09:30.347834 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ac3c13-c7f9-4411-ad91-c605fbfcef19 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:09:30.360946 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ac3c13-c7f9-4411-ad91-c605fbfcef19 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:09:30.365862 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a39584-f4e5-4354-a27e-bfbd16d926ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:09:30.366483 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a39584-f4e5-4354-a27e-bfbd16d926ee admin admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:09:30.366811 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a39584-f4e5-4354-a27e-bfbd16d926ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:09:30.367093 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a39584-f4e5-4354-a27e-bfbd16d926ee admin admin] PATH_INFO: `/v3/roles` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:09:30.367726 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a39584-f4e5-4354-a27e-bfbd16d926ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:09:30.369328 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a39584-f4e5-4354-a27e-bfbd16d926ee admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:09:30.380556 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1016/1996] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 10 12:09:30 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 12:09:30.381292 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a39584-f4e5-4354-a27e-bfbd16d926ee admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:09:30.391284 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a39584-f4e5-4354-a27e-bfbd16d926ee admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:09:30.396674 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d908c217-7c96-438d-9790-fb3604db8610 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:09:30.397287 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d908c217-7c96-438d-9790-fb3604db8610 admin admin] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:09:30.397596 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d908c217-7c96-438d-9790-fb3604db8610 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:09:30.397872 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d908c217-7c96-438d-9790-fb3604db8610 admin admin] PATH_INFO: `/v3/projects` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:09:30.398506 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d908c217-7c96-438d-9790-fb3604db8610 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:09:30.400524 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d908c217-7c96-438d-9790-fb3604db8610 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:09:30.422710 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 981/1997] 10.222.0.46 () {60 vars in 1153 bytes} [Mon Oct 10 12:09:30 2022] POST /identity/v3/projects => generated 373 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 12:09:30.423533 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa67e64-3174-422a-857a-593e349bdbf8 admin admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:09:30.494579 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa67e64-3174-422a-857a-593e349bdbf8 admin admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:09:30.499752 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc424ddb-bdc5-4ee9-93b3-bc75c24e4a56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:09:30.500380 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc424ddb-bdc5-4ee9-93b3-bc75c24e4a56 admin admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:09:30.500665 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc424ddb-bdc5-4ee9-93b3-bc75c24e4a56 admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:09:30.500951 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc424ddb-bdc5-4ee9-93b3-bc75c24e4a56 admin admin] PATH_INFO: `/v3/roles` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:09:30.501613 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc424ddb-bdc5-4ee9-93b3-bc75c24e4a56 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:09:30.503369 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc424ddb-bdc5-4ee9-93b3-bc75c24e4a56 admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:09:30.515018 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 982/1998] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 10 12:09:30 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 12:09:30.520647 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc424ddb-bdc5-4ee9-93b3-bc75c24e4a56 admin admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:09:30.526478 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc424ddb-bdc5-4ee9-93b3-bc75c24e4a56 admin admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:09:30.531414 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff312a88-37f1-452a-9b28-db3cdec3be2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:09:30.532039 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff312a88-37f1-452a-9b28-db3cdec3be2f admin admin] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:09:30.532325 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff312a88-37f1-452a-9b28-db3cdec3be2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:09:30.532604 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff312a88-37f1-452a-9b28-db3cdec3be2f admin admin] PATH_INFO: `/v3/users` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:09:30.533250 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff312a88-37f1-452a-9b28-db3cdec3be2f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:09:30.535360 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff312a88-37f1-452a-9b28-db3cdec3be2f admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:09:30.552414 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1017/1999] 10.222.0.46 () {60 vars in 1153 bytes} [Mon Oct 10 12:09:30 2022] POST /identity/v3/projects => generated 405 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 12:09:30.553262 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d908c217-7c96-438d-9790-fb3604db8610 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:09:30.618473 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d908c217-7c96-438d-9790-fb3604db8610 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:09:30.623765 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d56d0d-7381-4ac1-8289-cd0f305fdbbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:09:30.624450 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d56d0d-7381-4ac1-8289-cd0f305fdbbb admin admin] REQUEST_METHOD: `PUT` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:09:30.624796 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d56d0d-7381-4ac1-8289-cd0f305fdbbb admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:09:30.625097 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d56d0d-7381-4ac1-8289-cd0f305fdbbb admin admin] PATH_INFO: `/v3/projects/b8932e30ceda436896269ff0e8aae42e/users/2626b918b63b40d287fcf54a73b18be2/roles/b2cb2a6e818e47338ce071e6fedfc673` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:09:30.642511 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d56d0d-7381-4ac1-8289-cd0f305fdbbb admin admin] RBAC: Authorizing `identity:create_grant(project_id=b8932e30ceda436896269ff0e8aae42e, user_id=2626b918b63b40d287fcf54a73b18be2, role_id=b2cb2a6e818e47338ce071e6fedfc673)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:09:30.645265 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d56d0d-7381-4ac1-8289-cd0f305fdbbb admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:09:30.698201 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 983/2000] 10.222.0.46 () {60 vars in 1144 bytes} [Mon Oct 10 12:09:30 2022] POST /identity/v3/users => generated 287 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 12:09:30.699791 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff312a88-37f1-452a-9b28-db3cdec3be2f admin admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:09:30.718309 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff312a88-37f1-452a-9b28-db3cdec3be2f admin admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:09:30.723623 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1018/2001] 10.222.0.46 () {60 vars in 1483 bytes} [Mon Oct 10 12:09:30 2022] PUT /identity/v3/projects/b8932e30ceda436896269ff0e8aae42e/users/2626b918b63b40d287fcf54a73b18be2/roles/b2cb2a6e818e47338ce071e6fedfc673 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:09:30.725295 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d56d0d-7381-4ac1-8289-cd0f305fdbbb admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:09:30.809359 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7032710e-608f-406c-9dc0-465d6d751a96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:09:30.809591 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d56d0d-7381-4ac1-8289-cd0f305fdbbb admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:09:30.810055 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7032710e-608f-406c-9dc0-465d6d751a96 admin admin] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:09:30.810266 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7032710e-608f-406c-9dc0-465d6d751a96 admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:09:30.810536 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7032710e-608f-406c-9dc0-465d6d751a96 admin admin] PATH_INFO: `/v3/users` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:09:30.811244 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7032710e-608f-406c-9dc0-465d6d751a96 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:09:30.813167 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7032710e-608f-406c-9dc0-465d6d751a96 admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:09:30.814268 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b520b1-19f9-4d52-9aee-5c723e2142da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:09:30.815079 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b520b1-19f9-4d52-9aee-5c723e2142da admin admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:09:30.815374 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b520b1-19f9-4d52-9aee-5c723e2142da admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:09:30.815635 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b520b1-19f9-4d52-9aee-5c723e2142da admin admin] PATH_INFO: `/v3/roles` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:09:30.816299 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b520b1-19f9-4d52-9aee-5c723e2142da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:09:30.818120 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b520b1-19f9-4d52-9aee-5c723e2142da admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:09:30.829738 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1019/2002] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 10 12:09:30 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 12:09:30.830918 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff14f75e-5c0f-4f51-87ce-602b53699e83 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:09:30.836351 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff14f75e-5c0f-4f51-87ce-602b53699e83 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:09:30.836351 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff14f75e-5c0f-4f51-87ce-602b53699e83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:09:30.861933 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-ff14f75e-5c0f-4f51-87ce-602b53699e83 None None] MFA Rules not processed for user `2626b918b63b40d287fcf54a73b18be2`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:09:30.863847 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-ff14f75e-5c0f-4f51-87ce-602b53699e83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:09:30.912081 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 984/2003] 10.222.0.46 () {60 vars in 1144 bytes} [Mon Oct 10 12:09:30 2022] POST /identity/v3/users => generated 303 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 12:09:30.913028 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7032710e-608f-406c-9dc0-465d6d751a96 admin admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:09:30.928246 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7032710e-608f-406c-9dc0-465d6d751a96 admin admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:09:30.941329 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45ef74d-8363-4524-8094-2f0857905a33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:09:30.941329 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45ef74d-8363-4524-8094-2f0857905a33 admin admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:09:30.941329 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45ef74d-8363-4524-8094-2f0857905a33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:09:30.941329 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45ef74d-8363-4524-8094-2f0857905a33 admin admin] PATH_INFO: `/v3/roles` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:09:30.941935 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45ef74d-8363-4524-8094-2f0857905a33 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:09:30.941935 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45ef74d-8363-4524-8094-2f0857905a33 admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:09:30.944947 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:09:30.948862 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1020/2004] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:09:30 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:09:30.950386 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff14f75e-5c0f-4f51-87ce-602b53699e83 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:09:30.960442 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 985/2005] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 10 12:09:30 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 12:09:30.960442 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45ef74d-8363-4524-8094-2f0857905a33 admin admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:09:30.971293 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff14f75e-5c0f-4f51-87ce-602b53699e83 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:09:30.974488 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45ef74d-8363-4524-8094-2f0857905a33 admin admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:09:30.980466 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0360e4f-bb95-491b-ae8b-0a7b393f5f47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:09:30.980726 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15084f44-c818-4a04-95de-e380a255426a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:09:30.981488 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0360e4f-bb95-491b-ae8b-0a7b393f5f47 admin admin] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:09:30.981760 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15084f44-c818-4a04-95de-e380a255426a admin admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:09:30.982100 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0360e4f-bb95-491b-ae8b-0a7b393f5f47 admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:09:30.982205 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15084f44-c818-4a04-95de-e380a255426a admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:09:30.982388 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0360e4f-bb95-491b-ae8b-0a7b393f5f47 admin admin] PATH_INFO: `/v3/projects` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:09:30.982699 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15084f44-c818-4a04-95de-e380a255426a admin admin] PATH_INFO: `/v3/roles` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:09:30.983064 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0360e4f-bb95-491b-ae8b-0a7b393f5f47 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:09:30.983901 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15084f44-c818-4a04-95de-e380a255426a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:09:30.985073 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0360e4f-bb95-491b-ae8b-0a7b393f5f47 admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:09:30.986597 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15084f44-c818-4a04-95de-e380a255426a admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:09:31.004545 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1021/2006] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 10 12:09:30 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 12:09:31.006473 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15084f44-c818-4a04-95de-e380a255426a admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:09:31.020847 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15084f44-c818-4a04-95de-e380a255426a admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:09:31.026353 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ff5ebd-b656-4d79-a505-ca5081c0bee7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:09:31.027234 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ff5ebd-b656-4d79-a505-ca5081c0bee7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:09:31.027520 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ff5ebd-b656-4d79-a505-ca5081c0bee7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:09:31.027894 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ff5ebd-b656-4d79-a505-ca5081c0bee7 admin admin] PATH_INFO: `/v3/projects/470b821036534bbcb8b75f976560fa95/users/c281c8a4584c4160af9884c10a7cba18/roles/b2cb2a6e818e47338ce071e6fedfc673` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:09:31.033707 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:09:31.051957 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ff5ebd-b656-4d79-a505-ca5081c0bee7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=470b821036534bbcb8b75f976560fa95, user_id=c281c8a4584c4160af9884c10a7cba18, role_id=b2cb2a6e818e47338ce071e6fedfc673)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:09:31.054648 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ff5ebd-b656-4d79-a505-ca5081c0bee7 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:09:31.129745 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 986/2007] 10.222.0.46 () {60 vars in 1153 bytes} [Mon Oct 10 12:09:30 2022] POST /identity/v3/projects => generated 375 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 12:09:31.130420 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0360e4f-bb95-491b-ae8b-0a7b393f5f47 admin admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:09:31.159266 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1022/2008] 10.222.0.46 () {60 vars in 1483 bytes} [Mon Oct 10 12:09:31 2022] PUT /identity/v3/projects/470b821036534bbcb8b75f976560fa95/users/c281c8a4584c4160af9884c10a7cba18/roles/b2cb2a6e818e47338ce071e6fedfc673 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:09:31.161232 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ff5ebd-b656-4d79-a505-ca5081c0bee7 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:09:31.214085 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0360e4f-bb95-491b-ae8b-0a7b393f5f47 admin admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:09:31.244755 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ff5ebd-b656-4d79-a505-ca5081c0bee7 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:09:31.252618 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b90514-78f7-42df-a3e0-b6c394dde8f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:09:31.253640 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b90514-78f7-42df-a3e0-b6c394dde8f7 admin admin] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:09:31.254159 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b90514-78f7-42df-a3e0-b6c394dde8f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:09:31.254613 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b90514-78f7-42df-a3e0-b6c394dde8f7 admin admin] PATH_INFO: `/v3/users` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:09:31.259487 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b90514-78f7-42df-a3e0-b6c394dde8f7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:09:31.259487 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b90514-78f7-42df-a3e0-b6c394dde8f7 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:09:31.308647 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09831c5-b703-4d9f-ac68-438b5a0364e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:09:31.309288 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09831c5-b703-4d9f-ac68-438b5a0364e5 admin admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:09:31.309567 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09831c5-b703-4d9f-ac68-438b5a0364e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:09:31.309842 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09831c5-b703-4d9f-ac68-438b5a0364e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:09:31.310509 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09831c5-b703-4d9f-ac68-438b5a0364e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:09:31.312173 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09831c5-b703-4d9f-ac68-438b5a0364e5 admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:09:31.332263 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 987/2009] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 10 12:09:31 2022] GET /identity/v3/roles => generated 1420 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 12:09:31.333470 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c431fa5-5ab3-45ea-9de2-a8459ef86423 None None] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:09:31.333651 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c431fa5-5ab3-45ea-9de2-a8459ef86423 None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:09:31.333903 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c431fa5-5ab3-45ea-9de2-a8459ef86423 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:09:31.368464 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.auth.core [None req-7c431fa5-5ab3-45ea-9de2-a8459ef86423 None None] MFA Rules not processed for user `c281c8a4584c4160af9884c10a7cba18`. Rule list: `[]` (Enabled: `True`). {{(pid=81676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:09:31.370027 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-7c431fa5-5ab3-45ea-9de2-a8459ef86423 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:09:31.401595 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1023/2010] 10.222.0.46 () {60 vars in 1144 bytes} [Mon Oct 10 12:09:31 2022] POST /identity/v3/users => generated 294 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 12:09:31.402525 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b90514-78f7-42df-a3e0-b6c394dde8f7 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:09:31.417934 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b90514-78f7-42df-a3e0-b6c394dde8f7 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:09:31.426183 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9e150d-af9d-4711-a9ca-3cfe96da396a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:09:31.427382 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9e150d-af9d-4711-a9ca-3cfe96da396a admin admin] REQUEST_METHOD: `PUT` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:09:31.427889 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9e150d-af9d-4711-a9ca-3cfe96da396a admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:09:31.428388 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9e150d-af9d-4711-a9ca-3cfe96da396a admin admin] PATH_INFO: `/v3/projects/54e5eb205d3747dc9fad7bb19cee5e0f/users/e18f46ebbeeb40979b478689eec909f0/roles/b2cb2a6e818e47338ce071e6fedfc673` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:09:31.453386 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9e150d-af9d-4711-a9ca-3cfe96da396a admin admin] RBAC: Authorizing `identity:create_grant(project_id=54e5eb205d3747dc9fad7bb19cee5e0f, user_id=e18f46ebbeeb40979b478689eec909f0, role_id=b2cb2a6e818e47338ce071e6fedfc673)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:09:31.456218 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9e150d-af9d-4711-a9ca-3cfe96da396a admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:09:31.459918 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 988/2011] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:09:31 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:09:31.461468 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c431fa5-5ab3-45ea-9de2-a8459ef86423 None None] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:09:31.482112 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c431fa5-5ab3-45ea-9de2-a8459ef86423 None None] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:09:31.490351 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd2f438-c202-4473-bceb-d3b83e0afaf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:09:31.491447 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd2f438-c202-4473-bceb-d3b83e0afaf1 admin admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:09:31.491947 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd2f438-c202-4473-bceb-d3b83e0afaf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:09:31.492442 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd2f438-c202-4473-bceb-d3b83e0afaf1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:09:31.493642 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd2f438-c202-4473-bceb-d3b83e0afaf1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:09:31.496751 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd2f438-c202-4473-bceb-d3b83e0afaf1 admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:09:31.520181 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 989/2012] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 10 12:09:31 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 12:09:31.520181 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd2f438-c202-4473-bceb-d3b83e0afaf1 admin admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:09:31.520963 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-3dd2f438-c202-4473-bceb-d3b83e0afaf1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:09:31.588949 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd2f438-c202-4473-bceb-d3b83e0afaf1 admin admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:09:31.593428 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5856cfb4-6fcf-448f-91df-a49e9609683e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3ae093470484b93b23dcfae1657df9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad3772f7b6a47c28b7ba37a9316a3c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:09:31.594073 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5856cfb4-6fcf-448f-91df-a49e9609683e service cinder] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:09:31.594355 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5856cfb4-6fcf-448f-91df-a49e9609683e service cinder] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:09:31.594629 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5856cfb4-6fcf-448f-91df-a49e9609683e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:09:31.605075 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5856cfb4-6fcf-448f-91df-a49e9609683e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:09:31.606541 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5856cfb4-6fcf-448f-91df-a49e9609683e service cinder] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:09:31.607639 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1024/2013] 10.222.0.46 () {60 vars in 1483 bytes} [Mon Oct 10 12:09:31 2022] PUT /identity/v3/projects/54e5eb205d3747dc9fad7bb19cee5e0f/users/e18f46ebbeeb40979b478689eec909f0/roles/b2cb2a6e818e47338ce071e6fedfc673 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:09:31.608755 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9e150d-af9d-4711-a9ca-3cfe96da396a admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:09:31.617788 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 990/2014] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 10 12:09:31 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:09:31.619166 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6902f25c-50af-4a3c-8659-87e4aae1e286 None None] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:09:31.619454 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6902f25c-50af-4a3c-8659-87e4aae1e286 None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:09:31.619844 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6902f25c-50af-4a3c-8659-87e4aae1e286 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:09:31.644179 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.auth.core [None req-6902f25c-50af-4a3c-8659-87e4aae1e286 None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:09:31.646081 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-6902f25c-50af-4a3c-8659-87e4aae1e286 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:09:31.698088 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9e150d-af9d-4711-a9ca-3cfe96da396a admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:09:31.704529 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59a5afa-0a5b-45dc-a8d6-c1a5facda373 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:09:31.706126 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59a5afa-0a5b-45dc-a8d6-c1a5facda373 admin admin] REQUEST_METHOD: `PUT` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:09:31.706573 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59a5afa-0a5b-45dc-a8d6-c1a5facda373 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:09:31.707799 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59a5afa-0a5b-45dc-a8d6-c1a5facda373 admin admin] PATH_INFO: `/v3/projects/b00f6a2a946f4af6a5506e5b483aa2b9/users/420d942055b34777bfbca8d9f684c3e3/roles/0d822a0674b6495fac623877a3b38025` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:09:31.731195 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59a5afa-0a5b-45dc-a8d6-c1a5facda373 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b00f6a2a946f4af6a5506e5b483aa2b9, user_id=420d942055b34777bfbca8d9f684c3e3, role_id=0d822a0674b6495fac623877a3b38025)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:09:31.738194 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59a5afa-0a5b-45dc-a8d6-c1a5facda373 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:09:31.738734 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:09:31.743931 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 991/2015] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:09:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:09:31.754968 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6902f25c-50af-4a3c-8659-87e4aae1e286 None None] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:09:31.757018 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-6902f25c-50af-4a3c-8659-87e4aae1e286 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:09:31.828866 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6902f25c-50af-4a3c-8659-87e4aae1e286 None None] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:09:31.833330 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb1b9ba-7e18-4cfd-9eb4-88d5214ee611 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58b1c7c85302421cba823ace4201861f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad3772f7b6a47c28b7ba37a9316a3c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:09:31.834016 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb1b9ba-7e18-4cfd-9eb4-88d5214ee611 service neutron] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:09:31.834283 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb1b9ba-7e18-4cfd-9eb4-88d5214ee611 service neutron] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:09:31.834559 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb1b9ba-7e18-4cfd-9eb4-88d5214ee611 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:09:31.845024 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb1b9ba-7e18-4cfd-9eb4-88d5214ee611 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:09:31.846488 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb1b9ba-7e18-4cfd-9eb4-88d5214ee611 service neutron] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:09:31.857420 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 992/2016] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 10 12:09:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 10 12:09:31.870374 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1025/2017] 10.222.0.46 () {60 vars in 1483 bytes} [Mon Oct 10 12:09:31 2022] PUT /identity/v3/projects/b00f6a2a946f4af6a5506e5b483aa2b9/users/420d942055b34777bfbca8d9f684c3e3/roles/0d822a0674b6495fac623877a3b38025 => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:09:31.874751 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb1b9ba-7e18-4cfd-9eb4-88d5214ee611 service neutron] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:09:31.885450 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb1b9ba-7e18-4cfd-9eb4-88d5214ee611 service neutron] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:09:31.955530 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f36d23-7e01-4582-80fa-5282f7a41c9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:09:31.956166 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f36d23-7e01-4582-80fa-5282f7a41c9f admin admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:09:31.956450 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f36d23-7e01-4582-80fa-5282f7a41c9f admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:09:31.956779 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f36d23-7e01-4582-80fa-5282f7a41c9f admin admin] PATH_INFO: `/v3/roles` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:09:31.957425 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f36d23-7e01-4582-80fa-5282f7a41c9f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:09:31.959114 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f36d23-7e01-4582-80fa-5282f7a41c9f admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:09:31.970883 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 993/2018] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 10 12:09:31 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 10 12:09:31.975155 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59a5afa-0a5b-45dc-a8d6-c1a5facda373 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:09:31.986471 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59a5afa-0a5b-45dc-a8d6-c1a5facda373 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:09:31.992486 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80506ec7-c5c4-4665-9648-4792b6ecb6bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:09:31.993187 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80506ec7-c5c4-4665-9648-4792b6ecb6bc admin admin] REQUEST_METHOD: `PUT` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:09:31.993469 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80506ec7-c5c4-4665-9648-4792b6ecb6bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:09:31.993914 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80506ec7-c5c4-4665-9648-4792b6ecb6bc admin admin] PATH_INFO: `/v3/projects/b00f6a2a946f4af6a5506e5b483aa2b9/users/420d942055b34777bfbca8d9f684c3e3/roles/0d822a0674b6495fac623877a3b38025` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:09:31.995909 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80506ec7-c5c4-4665-9648-4792b6ecb6bc admin admin] RBAC: Authorizing `identity:create_grant(project_id=b00f6a2a946f4af6a5506e5b483aa2b9, user_id=420d942055b34777bfbca8d9f684c3e3, role_id=0d822a0674b6495fac623877a3b38025)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:09:31.998833 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80506ec7-c5c4-4665-9648-4792b6ecb6bc admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:09:32.012255 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1026/2019] 10.222.0.46 () {60 vars in 1483 bytes} [Mon Oct 10 12:09:31 2022] PUT /identity/v3/projects/b00f6a2a946f4af6a5506e5b483aa2b9/users/420d942055b34777bfbca8d9f684c3e3/roles/0d822a0674b6495fac623877a3b38025 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:09:32.023121 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9fb1d6-8b23-4e9d-aa73-596e3e27bcb9 None None] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:09:32.023419 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9fb1d6-8b23-4e9d-aa73-596e3e27bcb9 None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:09:32.023758 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9fb1d6-8b23-4e9d-aa73-596e3e27bcb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:09:32.030254 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:09:32.076301 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.auth.core [None req-fa9fb1d6-8b23-4e9d-aa73-596e3e27bcb9 None None] MFA Rules not processed for user `420d942055b34777bfbca8d9f684c3e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:09:32.077740 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-fa9fb1d6-8b23-4e9d-aa73-596e3e27bcb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:09:32.190327 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 994/2020] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:09:32 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 10 12:09:32.197927 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80506ec7-c5c4-4665-9648-4792b6ecb6bc admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:09:32.199874 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-80506ec7-c5c4-4665-9648-4792b6ecb6bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:09:32.286830 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80506ec7-c5c4-4665-9648-4792b6ecb6bc admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:09:32.299611 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79e23da-636c-40da-8358-768cec6275bb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3ae093470484b93b23dcfae1657df9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad3772f7b6a47c28b7ba37a9316a3c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:09:32.300420 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79e23da-636c-40da-8358-768cec6275bb service cinder] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:09:32.301480 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79e23da-636c-40da-8358-768cec6275bb service cinder] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:09:32.302259 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79e23da-636c-40da-8358-768cec6275bb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:09:32.304799 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-a79e23da-636c-40da-8358-768cec6275bb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:09:32.405743 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79e23da-636c-40da-8358-768cec6275bb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:09:32.407308 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79e23da-636c-40da-8358-768cec6275bb service cinder] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:09:32.418322 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1027/2021] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 10 12:09:32 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 221 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:09:32.818622 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:INSERT=7,keystone:SELECT=93 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:09:32.981273 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:09:33.862719 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:09:34.102623 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7055d70e-02d1-4d44-afff-22507cd46269 None None] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:09:34.109133 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7055d70e-02d1-4d44-afff-22507cd46269 None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:09:34.109133 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7055d70e-02d1-4d44-afff-22507cd46269 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:09:34.137271 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.auth.core [None req-7055d70e-02d1-4d44-afff-22507cd46269 None None] MFA Rules not processed for user `e18f46ebbeeb40979b478689eec909f0`. Rule list: `[]` (Enabled: `True`). {{(pid=81676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:09:34.138814 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-7055d70e-02d1-4d44-afff-22507cd46269 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:09:34.262913 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 995/2022] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:09:34 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:09:34.267662 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79e23da-636c-40da-8358-768cec6275bb service cinder] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:09:34.334715 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:09:34.363680 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79e23da-636c-40da-8358-768cec6275bb service cinder] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:09:34.368784 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43281e19-c48e-46e5-981b-cf4ec2519915 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:09:34.369486 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43281e19-c48e-46e5-981b-cf4ec2519915 admin admin] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:09:34.369832 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43281e19-c48e-46e5-981b-cf4ec2519915 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:09:34.370196 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43281e19-c48e-46e5-981b-cf4ec2519915 admin admin] PATH_INFO: `/v3/projects` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:09:34.370993 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43281e19-c48e-46e5-981b-cf4ec2519915 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:09:34.373000 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43281e19-c48e-46e5-981b-cf4ec2519915 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:09:34.564709 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1028/2023] 10.222.0.46 () {60 vars in 1153 bytes} [Mon Oct 10 12:09:34 2022] POST /identity/v3/projects => generated 401 bytes in 294 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 12:09:34.567004 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7055d70e-02d1-4d44-afff-22507cd46269 None None] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:09:34.579286 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7055d70e-02d1-4d44-afff-22507cd46269 None None] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:09:34.654284 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:09:34.656659 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53289de-3b0f-43ec-9995-17aa05410685 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:09:34.657809 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53289de-3b0f-43ec-9995-17aa05410685 admin admin] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:09:34.658131 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53289de-3b0f-43ec-9995-17aa05410685 admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:09:34.658423 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53289de-3b0f-43ec-9995-17aa05410685 admin admin] PATH_INFO: `/v3/users` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:09:34.659164 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53289de-3b0f-43ec-9995-17aa05410685 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:09:34.661465 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53289de-3b0f-43ec-9995-17aa05410685 admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:09:34.855541 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43281e19-c48e-46e5-981b-cf4ec2519915 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:09:34.861283 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 996/2024] 10.222.0.46 () {60 vars in 1144 bytes} [Mon Oct 10 12:09:34 2022] POST /identity/v3/users => generated 307 bytes in 279 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 12:09:34.876743 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43281e19-c48e-46e5-981b-cf4ec2519915 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:09:34.884710 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652e182e-3366-4711-8629-7908512c1ecd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:09:34.885696 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652e182e-3366-4711-8629-7908512c1ecd admin admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:09:34.886162 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652e182e-3366-4711-8629-7908512c1ecd admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:09:34.886617 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652e182e-3366-4711-8629-7908512c1ecd admin admin] PATH_INFO: `/v3/roles` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:09:34.887756 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652e182e-3366-4711-8629-7908512c1ecd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:09:34.890826 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652e182e-3366-4711-8629-7908512c1ecd admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:09:34.914281 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1029/2025] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 10 12:09:34 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 12:09:34.918997 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53289de-3b0f-43ec-9995-17aa05410685 admin admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:09:34.936356 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53289de-3b0f-43ec-9995-17aa05410685 admin admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:09:34.939171 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5145ac60-2312-4329-b45d-888ec19b7c2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:09:34.940348 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5145ac60-2312-4329-b45d-888ec19b7c2f admin admin] REQUEST_METHOD: `PUT` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:09:34.940864 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5145ac60-2312-4329-b45d-888ec19b7c2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:09:34.941929 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5145ac60-2312-4329-b45d-888ec19b7c2f admin admin] PATH_INFO: `/v3/projects/d332089fa3ba434386a1255ff0fb12e5/users/bb866447b2694138a4098a3aeb55551a/roles/0d822a0674b6495fac623877a3b38025` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:09:34.982137 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5145ac60-2312-4329-b45d-888ec19b7c2f admin admin] RBAC: Authorizing `identity:create_grant(project_id=d332089fa3ba434386a1255ff0fb12e5, user_id=bb866447b2694138a4098a3aeb55551a, role_id=0d822a0674b6495fac623877a3b38025)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:09:34.988596 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5145ac60-2312-4329-b45d-888ec19b7c2f admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:09:35.121212 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 997/2026] 10.222.0.46 () {60 vars in 1483 bytes} [Mon Oct 10 12:09:34 2022] PUT /identity/v3/projects/d332089fa3ba434386a1255ff0fb12e5/users/bb866447b2694138a4098a3aeb55551a/roles/0d822a0674b6495fac623877a3b38025 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 10 12:09:35.125690 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652e182e-3366-4711-8629-7908512c1ecd admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:09:35.140027 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652e182e-3366-4711-8629-7908512c1ecd admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:09:35.212306 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc349c63-5837-409b-a105-980c9e2096b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:09:35.213021 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc349c63-5837-409b-a105-980c9e2096b6 admin admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:09:35.213375 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc349c63-5837-409b-a105-980c9e2096b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:09:35.213703 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc349c63-5837-409b-a105-980c9e2096b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:09:35.214442 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc349c63-5837-409b-a105-980c9e2096b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:09:35.216131 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc349c63-5837-409b-a105-980c9e2096b6 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:09:35.229492 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1030/2027] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 10 12:09:35 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 10 12:09:35.234867 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5145ac60-2312-4329-b45d-888ec19b7c2f admin admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:09:35.250844 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5145ac60-2312-4329-b45d-888ec19b7c2f admin admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:09:35.256136 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3955fc9-d2be-4a4d-918d-d6391956738c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:09:35.256895 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3955fc9-d2be-4a4d-918d-d6391956738c admin admin] REQUEST_METHOD: `PUT` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:09:35.257146 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3955fc9-d2be-4a4d-918d-d6391956738c admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:09:35.257569 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3955fc9-d2be-4a4d-918d-d6391956738c admin admin] PATH_INFO: `/v3/projects/d332089fa3ba434386a1255ff0fb12e5/users/bb866447b2694138a4098a3aeb55551a/roles/0d822a0674b6495fac623877a3b38025` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:09:35.259688 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3955fc9-d2be-4a4d-918d-d6391956738c admin admin] RBAC: Authorizing `identity:create_grant(project_id=d332089fa3ba434386a1255ff0fb12e5, user_id=bb866447b2694138a4098a3aeb55551a, role_id=0d822a0674b6495fac623877a3b38025)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:09:35.263220 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3955fc9-d2be-4a4d-918d-d6391956738c admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:09:35.288789 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 998/2028] 10.222.0.46 () {60 vars in 1483 bytes} [Mon Oct 10 12:09:35 2022] PUT /identity/v3/projects/d332089fa3ba434386a1255ff0fb12e5/users/bb866447b2694138a4098a3aeb55551a/roles/0d822a0674b6495fac623877a3b38025 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:09:36.411865 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc349c63-5837-409b-a105-980c9e2096b6 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:09:36.489613 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc349c63-5837-409b-a105-980c9e2096b6 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:09:36.548128 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100369db-894f-41e8-aea0-f36543662df0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3ae093470484b93b23dcfae1657df9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad3772f7b6a47c28b7ba37a9316a3c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:09:36.558896 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100369db-894f-41e8-aea0-f36543662df0 service cinder] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:09:36.558896 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100369db-894f-41e8-aea0-f36543662df0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:09:36.558896 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100369db-894f-41e8-aea0-f36543662df0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:09:36.564933 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:09:36.614994 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100369db-894f-41e8-aea0-f36543662df0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:09:36.616403 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100369db-894f-41e8-aea0-f36543662df0 service cinder] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:09:36.626430 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1031/2029] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 10 12:09:36 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 215 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 10 12:09:37.207023 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199b5507-8648-44c7-b3cd-41ca91ba526c None None] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:09:37.207938 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199b5507-8648-44c7-b3cd-41ca91ba526c None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:09:37.208253 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199b5507-8648-44c7-b3cd-41ca91ba526c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:09:37.252742 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.auth.core [None req-199b5507-8648-44c7-b3cd-41ca91ba526c None None] MFA Rules not processed for user `bb866447b2694138a4098a3aeb55551a`. Rule list: `[]` (Enabled: `True`). {{(pid=81676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:09:37.254364 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-199b5507-8648-44c7-b3cd-41ca91ba526c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:09:37.356214 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 999/2030] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:09:37 2022] POST /identity/v3/auth/tokens => generated 3345 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:09:38.077173 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125b2a9c-c5d8-4dca-b05f-db078c775873 None None] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:09:38.078047 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125b2a9c-c5d8-4dca-b05f-db078c775873 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:09:38.078465 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125b2a9c-c5d8-4dca-b05f-db078c775873 None None] PATH_INFO: `/` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:09:38.086561 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1032/2031] 10.222.0.46 () {54 vars in 921 bytes} [Mon Oct 10 12:09:38 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 12:09:38.090979 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e70bddc-4f61-4e6f-bc23-71014748cddb None None] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:09:38.091303 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e70bddc-4f61-4e6f-bc23-71014748cddb None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:09:38.091588 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e70bddc-4f61-4e6f-bc23-71014748cddb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:09:38.107302 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:09:38.128052 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.auth.core [None req-1e70bddc-4f61-4e6f-bc23-71014748cddb None None] MFA Rules not processed for user `8e8f9bb1776d47169ef7febb57c954cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:09:38.129977 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-1e70bddc-4f61-4e6f-bc23-71014748cddb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:09:38.193816 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125b2a9c-c5d8-4dca-b05f-db078c775873 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:09:38.225219 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1000/2032] 10.222.0.46 () {58 vars in 1018 bytes} [Mon Oct 10 12:09:38 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:09:38.232867 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e70bddc-4f61-4e6f-bc23-71014748cddb None None] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:09:38.234371 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-1e70bddc-4f61-4e6f-bc23-71014748cddb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:09:38.251766 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e70bddc-4f61-4e6f-bc23-71014748cddb None None] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:09:38.257395 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195b8bcf-819b-43e5-9608-db41fc58d4ab service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e8f9bb1776d47169ef7febb57c954cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad3772f7b6a47c28b7ba37a9316a3c1', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:09:38.258020 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195b8bcf-819b-43e5-9608-db41fc58d4ab service placement] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:09:38.258310 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195b8bcf-819b-43e5-9608-db41fc58d4ab service placement] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:09:38.258584 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195b8bcf-819b-43e5-9608-db41fc58d4ab service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:09:38.267945 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195b8bcf-819b-43e5-9608-db41fc58d4ab service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:09:38.270768 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195b8bcf-819b-43e5-9608-db41fc58d4ab service placement] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:09:38.282310 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1001/2033] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 10 12:09:38 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 10 12:09:38.285869 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125b2a9c-c5d8-4dca-b05f-db078c775873 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:09:38.290262 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08cc5e75-3b36-4293-aa08-536e70022740 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:09:38.290928 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cc5e75-3b36-4293-aa08-536e70022740 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:09:38.291206 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cc5e75-3b36-4293-aa08-536e70022740 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:09:38.291478 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cc5e75-3b36-4293-aa08-536e70022740 admin admin] PATH_INFO: `/v3/users/c281c8a4584c4160af9884c10a7cba18` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:09:38.292617 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08cc5e75-3b36-4293-aa08-536e70022740 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c281c8a4584c4160af9884c10a7cba18)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:09:38.294529 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08cc5e75-3b36-4293-aa08-536e70022740 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:09:38.568704 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-08cc5e75-3b36-4293-aa08-536e70022740 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c281c8a4584c4160af9884c10a7cba18'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:09:38.581455 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-08cc5e75-3b36-4293-aa08-536e70022740 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c281c8a4584c4160af9884c10a7cba18'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:09:38.583012 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-08cc5e75-3b36-4293-aa08-536e70022740 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c281c8a4584c4160af9884c10a7cba18'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:09:38.663552 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-08cc5e75-3b36-4293-aa08-536e70022740 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c281c8a4584c4160af9884c10a7cba18'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:09:38.695004 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-08cc5e75-3b36-4293-aa08-536e70022740 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c281c8a4584c4160af9884c10a7cba18'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:09:38.702357 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1033/2034] 10.222.0.46 () {58 vars in 1224 bytes} [Mon Oct 10 12:09:38 2022] DELETE /identity/v3/users/c281c8a4584c4160af9884c10a7cba18 => generated 0 bytes in 509 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:09:38.707076 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d05270-fb09-49a3-b39d-d8755a3d2654 None None] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:09:38.707566 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d05270-fb09-49a3-b39d-d8755a3d2654 None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:09:38.707914 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d05270-fb09-49a3-b39d-d8755a3d2654 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:09:38.751138 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.auth.core [None req-16d05270-fb09-49a3-b39d-d8755a3d2654 None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:09:38.752484 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-16d05270-fb09-49a3-b39d-d8755a3d2654 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:09:38.838850 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1002/2035] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:09:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:09:38.849255 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08cc5e75-3b36-4293-aa08-536e70022740 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:09:38.850769 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-08cc5e75-3b36-4293-aa08-536e70022740 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:09:38.909722 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:09:38.943781 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08cc5e75-3b36-4293-aa08-536e70022740 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:09:38.948646 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ee69ad-d04b-4b44-9a5d-145adc468f8f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58b1c7c85302421cba823ace4201861f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad3772f7b6a47c28b7ba37a9316a3c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:09:38.949275 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ee69ad-d04b-4b44-9a5d-145adc468f8f service neutron] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:09:38.949559 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ee69ad-d04b-4b44-9a5d-145adc468f8f service neutron] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:09:38.949842 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ee69ad-d04b-4b44-9a5d-145adc468f8f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:09:38.960799 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ee69ad-d04b-4b44-9a5d-145adc468f8f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:09:38.962379 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ee69ad-d04b-4b44-9a5d-145adc468f8f service neutron] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:09:38.974719 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1034/2036] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 10 12:09:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:09:39.024118 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d05270-fb09-49a3-b39d-d8755a3d2654 None None] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:09:39.025539 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-16d05270-fb09-49a3-b39d-d8755a3d2654 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:09:39.087473 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d05270-fb09-49a3-b39d-d8755a3d2654 None None] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:09:39.091055 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c72f8f-9c44-4def-8c7b-d128fd0d7ad2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3ae093470484b93b23dcfae1657df9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad3772f7b6a47c28b7ba37a9316a3c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:09:39.091664 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c72f8f-9c44-4def-8c7b-d128fd0d7ad2 service cinder] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:09:39.091966 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c72f8f-9c44-4def-8c7b-d128fd0d7ad2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:09:39.092218 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c72f8f-9c44-4def-8c7b-d128fd0d7ad2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:09:39.093929 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-d5c72f8f-9c44-4def-8c7b-d128fd0d7ad2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:09:39.170929 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c72f8f-9c44-4def-8c7b-d128fd0d7ad2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:09:39.172485 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c72f8f-9c44-4def-8c7b-d128fd0d7ad2 service cinder] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:09:39.183715 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1003/2037] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 10 12:09:39 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:09:39.216944 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ee69ad-d04b-4b44-9a5d-145adc468f8f service neutron] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:09:39.218526 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-b2ee69ad-d04b-4b44-9a5d-145adc468f8f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:09:39.247432 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ee69ad-d04b-4b44-9a5d-145adc468f8f service neutron] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:09:39.259122 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8503120-08d8-434d-adf6-22e0734634a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:09:39.259859 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8503120-08d8-434d-adf6-22e0734634a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:09:39.260097 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8503120-08d8-434d-adf6-22e0734634a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:09:39.260200 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:09:39.260852 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8503120-08d8-434d-adf6-22e0734634a2 admin admin] PATH_INFO: `/v3/users/e18f46ebbeeb40979b478689eec909f0` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:09:39.266103 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8503120-08d8-434d-adf6-22e0734634a2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e18f46ebbeeb40979b478689eec909f0)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:09:39.266103 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8503120-08d8-434d-adf6-22e0734634a2 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:09:39.610125 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-a8503120-08d8-434d-adf6-22e0734634a2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e18f46ebbeeb40979b478689eec909f0'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:09:39.617164 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-a8503120-08d8-434d-adf6-22e0734634a2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e18f46ebbeeb40979b478689eec909f0'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:09:39.617766 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-a8503120-08d8-434d-adf6-22e0734634a2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e18f46ebbeeb40979b478689eec909f0'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:09:39.811014 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-a8503120-08d8-434d-adf6-22e0734634a2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e18f46ebbeeb40979b478689eec909f0'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:09:39.818943 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c72f8f-9c44-4def-8c7b-d128fd0d7ad2 service cinder] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:09:39.820612 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-d5c72f8f-9c44-4def-8c7b-d128fd0d7ad2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:09:39.840408 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c72f8f-9c44-4def-8c7b-d128fd0d7ad2 service cinder] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:09:39.848582 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-a8503120-08d8-434d-adf6-22e0734634a2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e18f46ebbeeb40979b478689eec909f0'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:09:39.849348 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a4ca6d-91fa-44d3-ac89-bf175fa88562 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:09:39.850203 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a4ca6d-91fa-44d3-ac89-bf175fa88562 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:09:39.850741 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a4ca6d-91fa-44d3-ac89-bf175fa88562 admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:09:39.851187 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a4ca6d-91fa-44d3-ac89-bf175fa88562 admin admin] PATH_INFO: `/v3/projects/470b821036534bbcb8b75f976560fa95` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:09:39.853371 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a4ca6d-91fa-44d3-ac89-bf175fa88562 admin admin] RBAC: Authorizing `identity:delete_project(project_id=470b821036534bbcb8b75f976560fa95)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:09:39.854986 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a4ca6d-91fa-44d3-ac89-bf175fa88562 admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:09:39.867965 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1035/2038] 10.222.0.46 () {58 vars in 1224 bytes} [Mon Oct 10 12:09:39 2022] DELETE /identity/v3/users/e18f46ebbeeb40979b478689eec909f0 => generated 0 bytes in 651 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:09:40.074703 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-10a4ca6d-91fa-44d3-ac89-bf175fa88562 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '470b821036534bbcb8b75f976560fa95', 'request_id': 'req-10a4ca6d-91fa-44d3-ac89-bf175fa88562'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:09:40.080999 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-10a4ca6d-91fa-44d3-ac89-bf175fa88562 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '470b821036534bbcb8b75f976560fa95', 'request_id': 'req-10a4ca6d-91fa-44d3-ac89-bf175fa88562'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:09:40.090737 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-10a4ca6d-91fa-44d3-ac89-bf175fa88562 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '470b821036534bbcb8b75f976560fa95', 'request_id': 'req-10a4ca6d-91fa-44d3-ac89-bf175fa88562'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:09:40.209309 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-10a4ca6d-91fa-44d3-ac89-bf175fa88562 admin admin] The token cache is being invalidate because project 470b821036534bbcb8b75f976560fa95 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81676) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 10 12:09:40.210177 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-10a4ca6d-91fa-44d3-ac89-bf175fa88562 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:09:40.211262 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8503120-08d8-434d-adf6-22e0734634a2 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:09:40.211420 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-10a4ca6d-91fa-44d3-ac89-bf175fa88562 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:09:40.213066 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-a8503120-08d8-434d-adf6-22e0734634a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:09:40.214144 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1004/2039] 10.222.0.46 () {58 vars in 1233 bytes} [Mon Oct 10 12:09:39 2022] DELETE /identity/v3/projects/470b821036534bbcb8b75f976560fa95 => generated 0 bytes in 396 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:09:40.221827 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e206c1-9331-420c-b293-77952649c6b1 None None] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:09:40.222322 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e206c1-9331-420c-b293-77952649c6b1 None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:09:40.222911 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e206c1-9331-420c-b293-77952649c6b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:09:40.269012 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.auth.core [None req-a3e206c1-9331-420c-b293-77952649c6b1 None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:09:40.270276 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-a3e206c1-9331-420c-b293-77952649c6b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:09:40.301249 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8503120-08d8-434d-adf6-22e0734634a2 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:09:40.352169 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1005/2040] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:09:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:09:40.360155 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b6a4c9-4f4b-4d94-afe6-d1b400a1765c None None] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:09:40.363195 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b6a4c9-4f4b-4d94-afe6-d1b400a1765c None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:09:40.363195 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b6a4c9-4f4b-4d94-afe6-d1b400a1765c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:09:40.386857 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.auth.core [None req-38b6a4c9-4f4b-4d94-afe6-d1b400a1765c None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:09:40.388136 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-38b6a4c9-4f4b-4d94-afe6-d1b400a1765c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:09:40.397248 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1006/2041] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:09:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 10 12:09:40.401869 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b6a4c9-4f4b-4d94-afe6-d1b400a1765c None None] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:09:40.411873 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b6a4c9-4f4b-4d94-afe6-d1b400a1765c None None] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:09:40.412100 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597ae583-898b-4406-b228-f06143817713 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:09:40.412749 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597ae583-898b-4406-b228-f06143817713 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:09:40.413434 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597ae583-898b-4406-b228-f06143817713 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:09:40.414124 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597ae583-898b-4406-b228-f06143817713 admin admin] PATH_INFO: `/v3/projects/54e5eb205d3747dc9fad7bb19cee5e0f` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:09:40.415645 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597ae583-898b-4406-b228-f06143817713 admin admin] RBAC: Authorizing `identity:delete_project(project_id=54e5eb205d3747dc9fad7bb19cee5e0f)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:09:40.416896 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45264124-74fb-423e-a8a6-bc2212f18d87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:09:40.417477 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45264124-74fb-423e-a8a6-bc2212f18d87 admin admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:09:40.428678 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597ae583-898b-4406-b228-f06143817713 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:09:40.428905 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45264124-74fb-423e-a8a6-bc2212f18d87 admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:09:40.428905 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45264124-74fb-423e-a8a6-bc2212f18d87 admin admin] PATH_INFO: `/v3/domains` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:09:40.428905 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45264124-74fb-423e-a8a6-bc2212f18d87 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 12:09:40.428905 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45264124-74fb-423e-a8a6-bc2212f18d87 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:09:40.428905 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45264124-74fb-423e-a8a6-bc2212f18d87 admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:09:40.438098 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1007/2042] 10.222.0.46 () {58 vars in 1153 bytes} [Mon Oct 10 12:09:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 12:09:40.439794 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45264124-74fb-423e-a8a6-bc2212f18d87 admin admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:09:40.455041 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45264124-74fb-423e-a8a6-bc2212f18d87 admin admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:09:40.460146 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45214cb9-f3a5-4e82-b853-8570e61beefb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:09:40.460924 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45214cb9-f3a5-4e82-b853-8570e61beefb admin admin] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:09:40.461207 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45214cb9-f3a5-4e82-b853-8570e61beefb admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:09:40.461481 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45214cb9-f3a5-4e82-b853-8570e61beefb admin admin] PATH_INFO: `/v3/projects` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:09:40.462134 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45214cb9-f3a5-4e82-b853-8570e61beefb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:09:40.464128 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45214cb9-f3a5-4e82-b853-8570e61beefb admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:09:40.596639 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-597ae583-898b-4406-b228-f06143817713 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '54e5eb205d3747dc9fad7bb19cee5e0f', 'request_id': 'req-597ae583-898b-4406-b228-f06143817713'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:09:40.608260 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-597ae583-898b-4406-b228-f06143817713 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '54e5eb205d3747dc9fad7bb19cee5e0f', 'request_id': 'req-597ae583-898b-4406-b228-f06143817713'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:09:40.621191 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-597ae583-898b-4406-b228-f06143817713 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '54e5eb205d3747dc9fad7bb19cee5e0f', 'request_id': 'req-597ae583-898b-4406-b228-f06143817713'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:09:40.625010 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1008/2043] 10.222.0.46 () {60 vars in 1153 bytes} [Mon Oct 10 12:09:40 2022] POST /identity/v3/projects => generated 383 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 12:09:40.632384 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45214cb9-f3a5-4e82-b853-8570e61beefb admin admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:09:40.665609 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:09:40.702022 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-597ae583-898b-4406-b228-f06143817713 admin admin] The token cache is being invalidate because project 54e5eb205d3747dc9fad7bb19cee5e0f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81677) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 10 12:09:40.702712 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-597ae583-898b-4406-b228-f06143817713 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:09:40.703526 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-597ae583-898b-4406-b228-f06143817713 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:09:40.709254 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1036/2044] 10.222.0.46 () {58 vars in 1233 bytes} [Mon Oct 10 12:09:40 2022] DELETE /identity/v3/projects/54e5eb205d3747dc9fad7bb19cee5e0f => generated 0 bytes in 495 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 10 12:09:40.710174 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597ae583-898b-4406-b228-f06143817713 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:09:40.711689 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-597ae583-898b-4406-b228-f06143817713 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:09:40.720847 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45214cb9-f3a5-4e82-b853-8570e61beefb admin admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:09:40.727455 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425fe4bf-48cc-42bd-960f-ff47ac80e472 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:09:40.728087 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425fe4bf-48cc-42bd-960f-ff47ac80e472 admin admin] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:09:40.728454 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425fe4bf-48cc-42bd-960f-ff47ac80e472 admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:09:40.728759 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425fe4bf-48cc-42bd-960f-ff47ac80e472 admin admin] PATH_INFO: `/v3/users` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:09:40.729416 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425fe4bf-48cc-42bd-960f-ff47ac80e472 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:09:40.731618 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425fe4bf-48cc-42bd-960f-ff47ac80e472 admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:09:40.786496 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597ae583-898b-4406-b228-f06143817713 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:09:40.791390 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4c6149-9979-45f4-9a6a-4ff727ce81b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:09:40.792035 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4c6149-9979-45f4-9a6a-4ff727ce81b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:09:40.794069 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4c6149-9979-45f4-9a6a-4ff727ce81b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:09:40.794069 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4c6149-9979-45f4-9a6a-4ff727ce81b1 admin admin] PATH_INFO: `/v3/users/70b34bc322c04ee287da162a1fe02a61` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:09:40.794069 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4c6149-9979-45f4-9a6a-4ff727ce81b1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=70b34bc322c04ee287da162a1fe02a61)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:09:40.795729 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4c6149-9979-45f4-9a6a-4ff727ce81b1 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:09:40.858182 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1009/2045] 10.222.0.46 () {60 vars in 1144 bytes} [Mon Oct 10 12:09:40 2022] POST /identity/v3/users => generated 292 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 10 12:09:40.859308 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425fe4bf-48cc-42bd-960f-ff47ac80e472 admin admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:09:40.860870 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-425fe4bf-48cc-42bd-960f-ff47ac80e472 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:09:40.873650 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425fe4bf-48cc-42bd-960f-ff47ac80e472 admin admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:09:40.878384 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012c58c4-1116-400a-be45-af101bdf8f15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:09:40.879065 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012c58c4-1116-400a-be45-af101bdf8f15 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:09:40.879348 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012c58c4-1116-400a-be45-af101bdf8f15 admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:09:40.879631 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012c58c4-1116-400a-be45-af101bdf8f15 admin admin] PATH_INFO: `/v3/users/bb866447b2694138a4098a3aeb55551a` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:09:40.880861 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012c58c4-1116-400a-be45-af101bdf8f15 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bb866447b2694138a4098a3aeb55551a)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:09:40.882832 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012c58c4-1116-400a-be45-af101bdf8f15 admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:09:40.895173 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:09:41.046986 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-6e4c6149-9979-45f4-9a6a-4ff727ce81b1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '70b34bc322c04ee287da162a1fe02a61'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:09:41.063254 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-6e4c6149-9979-45f4-9a6a-4ff727ce81b1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '70b34bc322c04ee287da162a1fe02a61'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:09:41.064313 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-6e4c6149-9979-45f4-9a6a-4ff727ce81b1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '70b34bc322c04ee287da162a1fe02a61'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:09:41.120047 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-012c58c4-1116-400a-be45-af101bdf8f15 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bb866447b2694138a4098a3aeb55551a'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:09:41.128795 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-012c58c4-1116-400a-be45-af101bdf8f15 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bb866447b2694138a4098a3aeb55551a'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:09:41.129744 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-012c58c4-1116-400a-be45-af101bdf8f15 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bb866447b2694138a4098a3aeb55551a'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:09:41.130517 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-6e4c6149-9979-45f4-9a6a-4ff727ce81b1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '70b34bc322c04ee287da162a1fe02a61'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:09:41.151199 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-6e4c6149-9979-45f4-9a6a-4ff727ce81b1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '70b34bc322c04ee287da162a1fe02a61'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:09:41.157579 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1037/2046] 10.222.0.46 () {58 vars in 1224 bytes} [Mon Oct 10 12:09:40 2022] DELETE /identity/v3/users/70b34bc322c04ee287da162a1fe02a61 => generated 0 bytes in 448 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:09:41.158426 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4c6149-9979-45f4-9a6a-4ff727ce81b1 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:09:41.160459 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-6e4c6149-9979-45f4-9a6a-4ff727ce81b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:09:41.229321 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-012c58c4-1116-400a-be45-af101bdf8f15 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bb866447b2694138a4098a3aeb55551a'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:09:41.231416 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4c6149-9979-45f4-9a6a-4ff727ce81b1 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:09:41.239131 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9870348b-a143-4f01-9fe4-03c95a861928 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:09:41.241389 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9870348b-a143-4f01-9fe4-03c95a861928 admin admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:09:41.242269 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9870348b-a143-4f01-9fe4-03c95a861928 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:09:41.242778 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9870348b-a143-4f01-9fe4-03c95a861928 admin admin] PATH_INFO: `/v3/roles` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:09:41.243932 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9870348b-a143-4f01-9fe4-03c95a861928 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:09:41.248963 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9870348b-a143-4f01-9fe4-03c95a861928 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:09:41.267746 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1038/2047] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 10 12:09:41 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 12:09:41.270987 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b01331-766d-4e4b-a0a6-725a2d89d30c None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:09:41.271630 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b01331-766d-4e4b-a0a6-725a2d89d30c None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:09:41.272117 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b01331-766d-4e4b-a0a6-725a2d89d30c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:09:41.276027 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-012c58c4-1116-400a-be45-af101bdf8f15 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bb866447b2694138a4098a3aeb55551a'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:09:41.294974 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1010/2048] 10.222.0.46 () {58 vars in 1224 bytes} [Mon Oct 10 12:09:40 2022] DELETE /identity/v3/users/bb866447b2694138a4098a3aeb55551a => generated 0 bytes in 432 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:09:41.294974 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012c58c4-1116-400a-be45-af101bdf8f15 admin admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:09:41.294974 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-012c58c4-1116-400a-be45-af101bdf8f15 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:09:41.305392 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-94b01331-766d-4e4b-a0a6-725a2d89d30c None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:09:41.307015 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-94b01331-766d-4e4b-a0a6-725a2d89d30c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:09:41.390161 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012c58c4-1116-400a-be45-af101bdf8f15 admin admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:09:41.396606 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eba0183-b3f9-47cd-b2e2-a36231a66e29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:09:41.397573 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eba0183-b3f9-47cd-b2e2-a36231a66e29 admin admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:09:41.398052 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eba0183-b3f9-47cd-b2e2-a36231a66e29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:09:41.398597 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eba0183-b3f9-47cd-b2e2-a36231a66e29 admin admin] PATH_INFO: `/v3/roles` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:09:41.399507 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eba0183-b3f9-47cd-b2e2-a36231a66e29 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:09:41.401322 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eba0183-b3f9-47cd-b2e2-a36231a66e29 admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:09:41.426737 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1011/2049] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 10 12:09:41 2022] GET /identity/v3/roles => generated 1420 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 12:09:41.429048 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eba0183-b3f9-47cd-b2e2-a36231a66e29 admin admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:09:41.447292 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eba0183-b3f9-47cd-b2e2-a36231a66e29 admin admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:09:41.453470 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050c3ead-2cb8-471d-937f-a3ec020e9f10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:09:41.457554 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050c3ead-2cb8-471d-937f-a3ec020e9f10 admin admin] REQUEST_METHOD: `PUT` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:09:41.457908 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050c3ead-2cb8-471d-937f-a3ec020e9f10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:09:41.458254 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050c3ead-2cb8-471d-937f-a3ec020e9f10 admin admin] PATH_INFO: `/v3/projects/647106e6a44d4e2ea3a19aa4ca09f8fa/users/42d58ead76bd4dedaef6dea6014f1b22/roles/b2cb2a6e818e47338ce071e6fedfc673` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:09:41.462086 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1039/2050] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:09:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:09:41.481778 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b01331-766d-4e4b-a0a6-725a2d89d30c None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:09:41.481778 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-94b01331-766d-4e4b-a0a6-725a2d89d30c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:09:41.508210 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050c3ead-2cb8-471d-937f-a3ec020e9f10 admin admin] RBAC: Authorizing `identity:create_grant(project_id=647106e6a44d4e2ea3a19aa4ca09f8fa, user_id=42d58ead76bd4dedaef6dea6014f1b22, role_id=b2cb2a6e818e47338ce071e6fedfc673)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:09:41.513902 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050c3ead-2cb8-471d-937f-a3ec020e9f10 admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:09:41.573500 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b01331-766d-4e4b-a0a6-725a2d89d30c None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:09:41.577289 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46b333c-6884-4869-a8ce-5db81455f733 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58b1c7c85302421cba823ace4201861f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad3772f7b6a47c28b7ba37a9316a3c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:09:41.577949 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46b333c-6884-4869-a8ce-5db81455f733 service neutron] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:09:41.578230 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46b333c-6884-4869-a8ce-5db81455f733 service neutron] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:09:41.578508 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46b333c-6884-4869-a8ce-5db81455f733 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:09:41.589423 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46b333c-6884-4869-a8ce-5db81455f733 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:09:41.590932 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46b333c-6884-4869-a8ce-5db81455f733 service neutron] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:09:41.596483 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1012/2051] 10.222.0.46 () {60 vars in 1483 bytes} [Mon Oct 10 12:09:41 2022] PUT /identity/v3/projects/647106e6a44d4e2ea3a19aa4ca09f8fa/users/42d58ead76bd4dedaef6dea6014f1b22/roles/b2cb2a6e818e47338ce071e6fedfc673 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 10 12:09:41.602944 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1040/2052] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 10 12:09:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:09:41.608759 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea46482-2a36-40ce-b922-e692adb6854a None None] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:09:41.609155 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea46482-2a36-40ce-b922-e692adb6854a None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:09:41.609497 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea46482-2a36-40ce-b922-e692adb6854a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:09:41.651785 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46b333c-6884-4869-a8ce-5db81455f733 service neutron] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:09:41.653499 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-b46b333c-6884-4869-a8ce-5db81455f733 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:09:41.663467 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.auth.core [None req-6ea46482-2a36-40ce-b922-e692adb6854a None None] MFA Rules not processed for user `42d58ead76bd4dedaef6dea6014f1b22`. Rule list: `[]` (Enabled: `True`). {{(pid=81676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:09:41.668172 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-6ea46482-2a36-40ce-b922-e692adb6854a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:09:41.680152 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46b333c-6884-4869-a8ce-5db81455f733 service neutron] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:09:41.710617 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:09:41.773792 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1013/2053] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:09:41 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 10 12:09:41.775005 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c8c39f-3583-4e98-a43d-dddf0d05b4fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:09:41.775865 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c8c39f-3583-4e98-a43d-dddf0d05b4fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:09:41.776252 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c8c39f-3583-4e98-a43d-dddf0d05b4fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:09:41.776623 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c8c39f-3583-4e98-a43d-dddf0d05b4fc admin admin] PATH_INFO: `/v3/projects/d332089fa3ba434386a1255ff0fb12e5` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:09:41.778166 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c8c39f-3583-4e98-a43d-dddf0d05b4fc admin admin] RBAC: Authorizing `identity:delete_project(project_id=d332089fa3ba434386a1255ff0fb12e5)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:09:41.778354 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea46482-2a36-40ce-b922-e692adb6854a None None] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:09:41.780929 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c8c39f-3583-4e98-a43d-dddf0d05b4fc admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:09:41.785905 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:09:41.792048 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea46482-2a36-40ce-b922-e692adb6854a None None] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:09:41.794880 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fcf5b01-990f-4507-bcbc-5be9cb69b253 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:09:41.795525 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcf5b01-990f-4507-bcbc-5be9cb69b253 admin admin] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:09:41.795818 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcf5b01-990f-4507-bcbc-5be9cb69b253 admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:09:41.796089 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcf5b01-990f-4507-bcbc-5be9cb69b253 admin admin] PATH_INFO: `/v3/projects` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:09:41.796734 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fcf5b01-990f-4507-bcbc-5be9cb69b253 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:09:41.798698 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fcf5b01-990f-4507-bcbc-5be9cb69b253 admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:09:41.929651 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1014/2054] 10.222.0.46 () {60 vars in 1153 bytes} [Mon Oct 10 12:09:41 2022] POST /identity/v3/projects => generated 385 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 10 12:09:41.930915 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fcf5b01-990f-4507-bcbc-5be9cb69b253 admin admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:09:41.959103 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-c7c8c39f-3583-4e98-a43d-dddf0d05b4fc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd332089fa3ba434386a1255ff0fb12e5', 'request_id': 'req-c7c8c39f-3583-4e98-a43d-dddf0d05b4fc'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:09:41.966384 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-c7c8c39f-3583-4e98-a43d-dddf0d05b4fc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd332089fa3ba434386a1255ff0fb12e5', 'request_id': 'req-c7c8c39f-3583-4e98-a43d-dddf0d05b4fc'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:09:41.978091 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-c7c8c39f-3583-4e98-a43d-dddf0d05b4fc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd332089fa3ba434386a1255ff0fb12e5', 'request_id': 'req-c7c8c39f-3583-4e98-a43d-dddf0d05b4fc'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:09:42.030448 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fcf5b01-990f-4507-bcbc-5be9cb69b253 admin admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:09:42.107110 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-c7c8c39f-3583-4e98-a43d-dddf0d05b4fc admin admin] The token cache is being invalidate because project d332089fa3ba434386a1255ff0fb12e5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81677) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 10 12:09:42.107763 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-c7c8c39f-3583-4e98-a43d-dddf0d05b4fc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:09:42.108895 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-c7c8c39f-3583-4e98-a43d-dddf0d05b4fc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:09:42.111460 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1041/2055] 10.222.0.46 () {58 vars in 1233 bytes} [Mon Oct 10 12:09:41 2022] DELETE /identity/v3/projects/d332089fa3ba434386a1255ff0fb12e5 => generated 0 bytes in 460 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:09:42.113317 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d4eddf-a2c5-49a6-a894-87cb19dfede1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:09:42.114408 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d4eddf-a2c5-49a6-a894-87cb19dfede1 admin admin] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:09:42.114995 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d4eddf-a2c5-49a6-a894-87cb19dfede1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:09:42.115507 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d4eddf-a2c5-49a6-a894-87cb19dfede1 admin admin] PATH_INFO: `/v3/users` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:09:42.117244 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d4eddf-a2c5-49a6-a894-87cb19dfede1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:09:42.118404 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f65435c-9e61-4946-a42b-0de29046ca04 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:09:42.118900 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f65435c-9e61-4946-a42b-0de29046ca04 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:09:42.120148 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f65435c-9e61-4946-a42b-0de29046ca04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:09:42.120971 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d4eddf-a2c5-49a6-a894-87cb19dfede1 admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:09:42.155443 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-4f65435c-9e61-4946-a42b-0de29046ca04 None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:09:42.157149 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-4f65435c-9e61-4946-a42b-0de29046ca04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:09:42.179526 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1042/2056] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:09:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 10 12:09:42.186916 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b5328e-2184-48b8-9b67-28caf2b7b61a None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:09:42.187260 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b5328e-2184-48b8-9b67-28caf2b7b61a None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:09:42.187543 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b5328e-2184-48b8-9b67-28caf2b7b61a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:09:42.214452 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-22b5328e-2184-48b8-9b67-28caf2b7b61a None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:09:42.216885 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-22b5328e-2184-48b8-9b67-28caf2b7b61a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:09:42.227957 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1043/2057] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:09:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:09:42.233307 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b5328e-2184-48b8-9b67-28caf2b7b61a None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:09:42.254153 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b5328e-2184-48b8-9b67-28caf2b7b61a None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:09:42.255293 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1015/2058] 10.222.0.46 () {60 vars in 1144 bytes} [Mon Oct 10 12:09:41 2022] POST /identity/v3/users => generated 293 bytes in 326 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 12:09:42.259622 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d4eddf-a2c5-49a6-a894-87cb19dfede1 admin admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:09:42.261803 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-d0d4eddf-a2c5-49a6-a894-87cb19dfede1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:09:42.263770 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f17181-ba89-46ce-8356-2301716721f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:09:42.264491 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f17181-ba89-46ce-8356-2301716721f6 admin admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:09:42.264848 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f17181-ba89-46ce-8356-2301716721f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:09:42.265347 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f17181-ba89-46ce-8356-2301716721f6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:09:42.266517 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f17181-ba89-46ce-8356-2301716721f6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 12:09:42.267084 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f17181-ba89-46ce-8356-2301716721f6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:09:42.270045 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f17181-ba89-46ce-8356-2301716721f6 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:09:42.288313 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d4eddf-a2c5-49a6-a894-87cb19dfede1 admin admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:09:42.289493 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9b8d85-9d0a-4218-816d-587cdacd64f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:09:42.290460 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9b8d85-9d0a-4218-816d-587cdacd64f2 admin admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:09:42.290953 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9b8d85-9d0a-4218-816d-587cdacd64f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:09:42.291414 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9b8d85-9d0a-4218-816d-587cdacd64f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:09:42.292088 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1044/2059] 10.222.0.46 () {58 vars in 1153 bytes} [Mon Oct 10 12:09:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 12:09:42.292621 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9b8d85-9d0a-4218-816d-587cdacd64f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:09:42.295875 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9b8d85-9d0a-4218-816d-587cdacd64f2 admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:09:42.298701 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f17181-ba89-46ce-8356-2301716721f6 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:09:42.313829 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1016/2060] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 10 12:09:42 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 12:09:42.314125 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f17181-ba89-46ce-8356-2301716721f6 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:09:42.315114 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9b8d85-9d0a-4218-816d-587cdacd64f2 admin admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:09:42.318794 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9766b5e-49e3-494e-baba-3ca2c94a1fcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:09:42.322970 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9766b5e-49e3-494e-baba-3ca2c94a1fcc admin admin] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:09:42.322970 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9766b5e-49e3-494e-baba-3ca2c94a1fcc admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:09:42.322970 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9766b5e-49e3-494e-baba-3ca2c94a1fcc admin admin] PATH_INFO: `/v3/projects` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:09:42.322970 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9766b5e-49e3-494e-baba-3ca2c94a1fcc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:09:42.322970 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9766b5e-49e3-494e-baba-3ca2c94a1fcc admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:09:42.326597 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9b8d85-9d0a-4218-816d-587cdacd64f2 admin admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:09:42.332111 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8428c7fc-87f1-4a3a-a97b-698954ea561a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:09:42.332829 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8428c7fc-87f1-4a3a-a97b-698954ea561a admin admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:09:42.333103 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8428c7fc-87f1-4a3a-a97b-698954ea561a admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:09:42.333376 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8428c7fc-87f1-4a3a-a97b-698954ea561a admin admin] PATH_INFO: `/v3/roles` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:09:42.334038 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8428c7fc-87f1-4a3a-a97b-698954ea561a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:09:42.335730 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8428c7fc-87f1-4a3a-a97b-698954ea561a admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:09:42.351127 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1017/2061] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 10 12:09:42 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 10 12:09:42.359847 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8428c7fc-87f1-4a3a-a97b-698954ea561a admin admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:09:42.372401 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8428c7fc-87f1-4a3a-a97b-698954ea561a admin admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:09:42.382693 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4dfef0-e0eb-47dd-a464-7eece281db6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:09:42.383403 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4dfef0-e0eb-47dd-a464-7eece281db6e admin admin] REQUEST_METHOD: `PUT` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:09:42.383849 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4dfef0-e0eb-47dd-a464-7eece281db6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:09:42.384136 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4dfef0-e0eb-47dd-a464-7eece281db6e admin admin] PATH_INFO: `/v3/projects/6eab5e80b43d47119bbc61441a423ccd/users/b6714679417841a6a1a0f71d1af255d0/roles/b2cb2a6e818e47338ce071e6fedfc673` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:09:42.403384 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4dfef0-e0eb-47dd-a464-7eece281db6e admin admin] RBAC: Authorizing `identity:create_grant(project_id=6eab5e80b43d47119bbc61441a423ccd, user_id=b6714679417841a6a1a0f71d1af255d0, role_id=b2cb2a6e818e47338ce071e6fedfc673)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:09:42.407365 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4dfef0-e0eb-47dd-a464-7eece281db6e admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:09:42.467306 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1045/2062] 10.222.0.46 () {60 vars in 1153 bytes} [Mon Oct 10 12:09:42 2022] POST /identity/v3/projects => generated 373 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 10 12:09:42.468774 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9766b5e-49e3-494e-baba-3ca2c94a1fcc admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:09:42.471722 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-a9766b5e-49e3-494e-baba-3ca2c94a1fcc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:09:42.489111 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1018/2063] 10.222.0.46 () {60 vars in 1483 bytes} [Mon Oct 10 12:09:42 2022] PUT /identity/v3/projects/6eab5e80b43d47119bbc61441a423ccd/users/b6714679417841a6a1a0f71d1af255d0/roles/b2cb2a6e818e47338ce071e6fedfc673 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:09:42.490897 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4dfef0-e0eb-47dd-a464-7eece281db6e admin admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:09:42.583356 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9766b5e-49e3-494e-baba-3ca2c94a1fcc admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:09:42.604391 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4dfef0-e0eb-47dd-a464-7eece281db6e admin admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:09:42.609789 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9538e4db-f71d-4a1a-bb0a-85b428ce8d8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:09:42.610458 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9538e4db-f71d-4a1a-bb0a-85b428ce8d8f admin admin] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:09:42.610784 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9538e4db-f71d-4a1a-bb0a-85b428ce8d8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:09:42.611067 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9538e4db-f71d-4a1a-bb0a-85b428ce8d8f admin admin] PATH_INFO: `/v3/users` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:09:42.611721 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9538e4db-f71d-4a1a-bb0a-85b428ce8d8f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:09:42.613770 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9538e4db-f71d-4a1a-bb0a-85b428ce8d8f admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:09:42.646525 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a23f8a-5dd4-4315-aac7-9d27177f727b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:09:42.647205 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a23f8a-5dd4-4315-aac7-9d27177f727b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:09:42.647778 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a23f8a-5dd4-4315-aac7-9d27177f727b admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:09:42.647778 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a23f8a-5dd4-4315-aac7-9d27177f727b admin admin] PATH_INFO: `/v3/projects/542cd4bec9244d86853ce11782bf5ead` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:09:42.648934 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a23f8a-5dd4-4315-aac7-9d27177f727b admin admin] RBAC: Authorizing `identity:delete_project(project_id=542cd4bec9244d86853ce11782bf5ead)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:09:42.650858 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a23f8a-5dd4-4315-aac7-9d27177f727b admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:09:42.676128 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:09:42.756280 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1019/2064] 10.222.0.46 () {60 vars in 1144 bytes} [Mon Oct 10 12:09:42 2022] POST /identity/v3/users => generated 287 bytes in 267 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 12:09:42.757562 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f4d7af-07c9-4262-a04c-173a2a019cdc None None] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:09:42.757883 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f4d7af-07c9-4262-a04c-173a2a019cdc None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:09:42.758161 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f4d7af-07c9-4262-a04c-173a2a019cdc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:09:42.788141 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.auth.core [None req-64f4d7af-07c9-4262-a04c-173a2a019cdc None None] MFA Rules not processed for user `b6714679417841a6a1a0f71d1af255d0`. Rule list: `[]` (Enabled: `True`). {{(pid=81676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:09:42.789376 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-64f4d7af-07c9-4262-a04c-173a2a019cdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:09:42.791165 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:09:42.810113 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-e3a23f8a-5dd4-4315-aac7-9d27177f727b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '542cd4bec9244d86853ce11782bf5ead', 'request_id': 'req-e3a23f8a-5dd4-4315-aac7-9d27177f727b'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:09:42.816213 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-e3a23f8a-5dd4-4315-aac7-9d27177f727b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '542cd4bec9244d86853ce11782bf5ead', 'request_id': 'req-e3a23f8a-5dd4-4315-aac7-9d27177f727b'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:09:42.822373 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-e3a23f8a-5dd4-4315-aac7-9d27177f727b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '542cd4bec9244d86853ce11782bf5ead', 'request_id': 'req-e3a23f8a-5dd4-4315-aac7-9d27177f727b'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:09:42.855633 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1020/2065] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:09:42 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:09:42.856648 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f4d7af-07c9-4262-a04c-173a2a019cdc None None] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:09:42.866882 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f4d7af-07c9-4262-a04c-173a2a019cdc None None] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:09:42.935597 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05026f0d-cdc7-46fd-82eb-321263e058c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:09:42.936335 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05026f0d-cdc7-46fd-82eb-321263e058c8 admin admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:09:42.936661 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05026f0d-cdc7-46fd-82eb-321263e058c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:09:42.936980 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05026f0d-cdc7-46fd-82eb-321263e058c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:09:42.937692 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05026f0d-cdc7-46fd-82eb-321263e058c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:09:42.939535 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05026f0d-cdc7-46fd-82eb-321263e058c8 admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:09:42.952280 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1021/2066] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 10 12:09:42 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 12:09:42.953186 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05026f0d-cdc7-46fd-82eb-321263e058c8 admin admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:09:42.964489 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05026f0d-cdc7-46fd-82eb-321263e058c8 admin admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:09:42.970412 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddec24a5-ff95-4139-b744-94858834e58c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:09:42.973737 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddec24a5-ff95-4139-b744-94858834e58c admin admin] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:09:42.973880 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-e3a23f8a-5dd4-4315-aac7-9d27177f727b admin admin] The token cache is being invalidate because project 542cd4bec9244d86853ce11782bf5ead was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81677) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 10 12:09:42.973975 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddec24a5-ff95-4139-b744-94858834e58c admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:09:42.974072 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-e3a23f8a-5dd4-4315-aac7-9d27177f727b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:09:42.974152 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddec24a5-ff95-4139-b744-94858834e58c admin admin] PATH_INFO: `/v3/projects` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:09:42.974152 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddec24a5-ff95-4139-b744-94858834e58c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:09:42.974330 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-e3a23f8a-5dd4-4315-aac7-9d27177f727b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:09:42.974695 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddec24a5-ff95-4139-b744-94858834e58c admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:09:42.975042 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1046/2067] 10.222.0.46 () {58 vars in 1233 bytes} [Mon Oct 10 12:09:42 2022] DELETE /identity/v3/projects/542cd4bec9244d86853ce11782bf5ead => generated 0 bytes in 507 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:09:42.976173 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a23f8a-5dd4-4315-aac7-9d27177f727b admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:09:42.977958 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-e3a23f8a-5dd4-4315-aac7-9d27177f727b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:09:42.992238 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:09:42.997659 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a23f8a-5dd4-4315-aac7-9d27177f727b admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:09:43.003081 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd6fb82-3cdb-4848-b7b7-8aed0e99341a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:09:43.004283 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd6fb82-3cdb-4848-b7b7-8aed0e99341a admin admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:09:43.004609 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd6fb82-3cdb-4848-b7b7-8aed0e99341a admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:09:43.004924 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd6fb82-3cdb-4848-b7b7-8aed0e99341a admin admin] PATH_INFO: `/v3/roles` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:09:43.005654 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd6fb82-3cdb-4848-b7b7-8aed0e99341a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:09:43.007414 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd6fb82-3cdb-4848-b7b7-8aed0e99341a admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:09:43.019946 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1047/2068] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 10 12:09:42 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 12:09:43.021450 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadbc92b-99ff-43c6-8736-829a2ebe5be0 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:09:43.021744 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadbc92b-99ff-43c6-8736-829a2ebe5be0 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:09:43.022025 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadbc92b-99ff-43c6-8736-829a2ebe5be0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:09:43.043542 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-aadbc92b-99ff-43c6-8736-829a2ebe5be0 None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:09:43.045041 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-aadbc92b-99ff-43c6-8736-829a2ebe5be0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:09:43.054278 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1048/2069] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:09:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:09:43.055234 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadbc92b-99ff-43c6-8736-829a2ebe5be0 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:09:43.072653 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadbc92b-99ff-43c6-8736-829a2ebe5be0 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:09:43.072653 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f6bc49-ab9e-496f-abaa-fa4e46a543a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:09:43.073230 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f6bc49-ab9e-496f-abaa-fa4e46a543a9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:09:43.073230 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f6bc49-ab9e-496f-abaa-fa4e46a543a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:09:43.073559 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f6bc49-ab9e-496f-abaa-fa4e46a543a9 admin admin] PATH_INFO: `/v3/projects/fc3245b0042447f58d7297d1b414e400/users/6c5eb0505d864d5f9c4caae177941f58/roles/b2cb2a6e818e47338ce071e6fedfc673` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:09:43.091521 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f6bc49-ab9e-496f-abaa-fa4e46a543a9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fc3245b0042447f58d7297d1b414e400, user_id=6c5eb0505d864d5f9c4caae177941f58, role_id=b2cb2a6e818e47338ce071e6fedfc673)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:09:43.094225 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f6bc49-ab9e-496f-abaa-fa4e46a543a9 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:09:43.099960 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:09:43.146129 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1022/2070] 10.222.0.46 () {60 vars in 1153 bytes} [Mon Oct 10 12:09:42 2022] POST /identity/v3/projects => generated 385 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 12:09:43.147498 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0c68a2-8add-407d-baf9-e5684a79da75 None None] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:09:43.147787 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0c68a2-8add-407d-baf9-e5684a79da75 None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:09:43.148081 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0c68a2-8add-407d-baf9-e5684a79da75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:09:43.168361 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.auth.core [None req-ab0c68a2-8add-407d-baf9-e5684a79da75 None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:09:43.169607 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-ab0c68a2-8add-407d-baf9-e5684a79da75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:09:43.220322 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1049/2071] 10.222.0.46 () {60 vars in 1483 bytes} [Mon Oct 10 12:09:43 2022] PUT /identity/v3/projects/fc3245b0042447f58d7297d1b414e400/users/6c5eb0505d864d5f9c4caae177941f58/roles/b2cb2a6e818e47338ce071e6fedfc673 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:09:43.221996 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f6bc49-ab9e-496f-abaa-fa4e46a543a9 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:09:43.224778 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-f5f6bc49-ab9e-496f-abaa-fa4e46a543a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:09:43.240710 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1023/2072] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:09:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:09:43.242159 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f92847-ef46-4360-906b-74bb2c451305 None None] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:09:43.242550 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f92847-ef46-4360-906b-74bb2c451305 None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:09:43.242959 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f92847-ef46-4360-906b-74bb2c451305 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:09:43.286598 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.auth.core [None req-84f92847-ef46-4360-906b-74bb2c451305 None None] MFA Rules not processed for user `6c5eb0505d864d5f9c4caae177941f58`. Rule list: `[]` (Enabled: `True`). {{(pid=81676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:09:43.289481 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-84f92847-ef46-4360-906b-74bb2c451305 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:09:43.312794 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f6bc49-ab9e-496f-abaa-fa4e46a543a9 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:09:43.321054 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca2e64a-f8c4-4fa7-8f9f-52a8773cdf91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:09:43.322159 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca2e64a-f8c4-4fa7-8f9f-52a8773cdf91 admin admin] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:09:43.322459 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca2e64a-f8c4-4fa7-8f9f-52a8773cdf91 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:09:43.323078 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca2e64a-f8c4-4fa7-8f9f-52a8773cdf91 admin admin] PATH_INFO: `/v3/users` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:09:43.324162 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca2e64a-f8c4-4fa7-8f9f-52a8773cdf91 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:09:43.328037 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca2e64a-f8c4-4fa7-8f9f-52a8773cdf91 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:09:43.369228 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1024/2073] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:09:43 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:09:43.370219 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f92847-ef46-4360-906b-74bb2c451305 None None] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:09:43.382053 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f92847-ef46-4360-906b-74bb2c451305 None None] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:09:43.387899 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d587f3b-4344-4f16-a632-8774b53989d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:09:43.388528 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d587f3b-4344-4f16-a632-8774b53989d3 admin admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:09:43.388812 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d587f3b-4344-4f16-a632-8774b53989d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:09:43.389096 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d587f3b-4344-4f16-a632-8774b53989d3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:09:43.389767 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d587f3b-4344-4f16-a632-8774b53989d3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 12:09:43.390076 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d587f3b-4344-4f16-a632-8774b53989d3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:09:43.391736 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d587f3b-4344-4f16-a632-8774b53989d3 admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:09:43.405396 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1025/2074] 10.222.0.46 () {58 vars in 1153 bytes} [Mon Oct 10 12:09:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 12:09:43.406267 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d587f3b-4344-4f16-a632-8774b53989d3 admin admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:09:43.417159 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d587f3b-4344-4f16-a632-8774b53989d3 admin admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:09:43.422174 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37010df-1995-4ee2-9186-23c3fb8e9f94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:09:43.422939 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37010df-1995-4ee2-9186-23c3fb8e9f94 admin admin] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:09:43.423254 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37010df-1995-4ee2-9186-23c3fb8e9f94 admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:09:43.423529 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37010df-1995-4ee2-9186-23c3fb8e9f94 admin admin] PATH_INFO: `/v3/projects` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:09:43.424180 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37010df-1995-4ee2-9186-23c3fb8e9f94 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:09:43.426656 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37010df-1995-4ee2-9186-23c3fb8e9f94 admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:09:43.442544 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1050/2075] 10.222.0.46 () {60 vars in 1144 bytes} [Mon Oct 10 12:09:43 2022] POST /identity/v3/users => generated 299 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 12:09:43.444199 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca2e64a-f8c4-4fa7-8f9f-52a8773cdf91 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:09:43.457391 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca2e64a-f8c4-4fa7-8f9f-52a8773cdf91 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:09:43.463858 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d89dc0c-bd91-451e-8be8-e612cc0101eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:09:43.464038 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d89dc0c-bd91-451e-8be8-e612cc0101eb admin admin] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:09:43.464253 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d89dc0c-bd91-451e-8be8-e612cc0101eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:09:43.464511 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d89dc0c-bd91-451e-8be8-e612cc0101eb admin admin] PATH_INFO: `/v3/projects` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:09:43.465170 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d89dc0c-bd91-451e-8be8-e612cc0101eb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:09:43.467239 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d89dc0c-bd91-451e-8be8-e612cc0101eb admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:09:43.481013 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:09:43.555889 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:09:43.564059 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1026/2076] 10.222.0.46 () {60 vars in 1153 bytes} [Mon Oct 10 12:09:43 2022] POST /identity/v3/projects => generated 371 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 12:09:43.566360 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37010df-1995-4ee2-9186-23c3fb8e9f94 admin admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:09:43.579877 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1051/2077] 10.222.0.46 () {60 vars in 1153 bytes} [Mon Oct 10 12:09:43 2022] POST /identity/v3/projects => generated 381 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 12:09:43.581767 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d89dc0c-bd91-451e-8be8-e612cc0101eb admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:09:43.683738 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37010df-1995-4ee2-9186-23c3fb8e9f94 admin admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:09:43.704832 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d89dc0c-bd91-451e-8be8-e612cc0101eb admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:09:43.710533 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea14307-8da5-4738-aa8a-ba37176caca0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:09:43.711273 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea14307-8da5-4738-aa8a-ba37176caca0 admin admin] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:09:43.711559 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea14307-8da5-4738-aa8a-ba37176caca0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:09:43.711841 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea14307-8da5-4738-aa8a-ba37176caca0 admin admin] PATH_INFO: `/v3/users` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:09:43.712496 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea14307-8da5-4738-aa8a-ba37176caca0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:09:43.714487 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea14307-8da5-4738-aa8a-ba37176caca0 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:09:43.773976 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa6513b-ffe5-4493-a9db-bfa23aadb48d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:09:43.774605 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa6513b-ffe5-4493-a9db-bfa23aadb48d admin admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:09:43.775001 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa6513b-ffe5-4493-a9db-bfa23aadb48d admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:09:43.775279 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa6513b-ffe5-4493-a9db-bfa23aadb48d admin admin] PATH_INFO: `/v3/roles` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:09:43.775941 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa6513b-ffe5-4493-a9db-bfa23aadb48d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:09:43.777790 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa6513b-ffe5-4493-a9db-bfa23aadb48d admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:09:43.793978 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1027/2078] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 10 12:09:43 2022] GET /identity/v3/roles => generated 1420 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 12:09:43.793978 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa6513b-ffe5-4493-a9db-bfa23aadb48d admin admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:09:43.804134 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa6513b-ffe5-4493-a9db-bfa23aadb48d admin admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:09:43.809050 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc68ce7-7fe5-4c56-b8d1-ea6886dc09ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:09:43.809668 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc68ce7-7fe5-4c56-b8d1-ea6886dc09ce admin admin] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:09:43.810035 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc68ce7-7fe5-4c56-b8d1-ea6886dc09ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:09:43.810309 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc68ce7-7fe5-4c56-b8d1-ea6886dc09ce admin admin] PATH_INFO: `/v3/users` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:09:43.810981 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc68ce7-7fe5-4c56-b8d1-ea6886dc09ce admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:09:43.813933 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc68ce7-7fe5-4c56-b8d1-ea6886dc09ce admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:09:43.867829 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1052/2079] 10.222.0.46 () {60 vars in 1144 bytes} [Mon Oct 10 12:09:43 2022] POST /identity/v3/users => generated 292 bytes in 284 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 12:09:43.867829 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea14307-8da5-4738-aa8a-ba37176caca0 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:09:43.884369 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea14307-8da5-4738-aa8a-ba37176caca0 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:09:43.893222 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312b1ad4-202c-4164-9271-1f0823afef41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:09:43.893222 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312b1ad4-202c-4164-9271-1f0823afef41 admin admin] REQUEST_METHOD: `PUT` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:09:43.893222 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312b1ad4-202c-4164-9271-1f0823afef41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:09:43.893222 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312b1ad4-202c-4164-9271-1f0823afef41 admin admin] PATH_INFO: `/v3/projects/8e81b11a2b804a809894c8211c42d278/users/d93c36f2d1c1425d88c39e800c205066/roles/0d822a0674b6495fac623877a3b38025` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:09:43.909886 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312b1ad4-202c-4164-9271-1f0823afef41 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8e81b11a2b804a809894c8211c42d278, user_id=d93c36f2d1c1425d88c39e800c205066, role_id=0d822a0674b6495fac623877a3b38025)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:09:43.912576 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312b1ad4-202c-4164-9271-1f0823afef41 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:09:44.004663 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1028/2080] 10.222.0.46 () {60 vars in 1144 bytes} [Mon Oct 10 12:09:43 2022] POST /identity/v3/users => generated 291 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 12:09:44.004663 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc68ce7-7fe5-4c56-b8d1-ea6886dc09ce admin admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:09:44.004946 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1053/2081] 10.222.0.46 () {60 vars in 1483 bytes} [Mon Oct 10 12:09:43 2022] PUT /identity/v3/projects/8e81b11a2b804a809894c8211c42d278/users/d93c36f2d1c1425d88c39e800c205066/roles/0d822a0674b6495fac623877a3b38025 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:09:44.007122 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312b1ad4-202c-4164-9271-1f0823afef41 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:09:44.019440 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc68ce7-7fe5-4c56-b8d1-ea6886dc09ce admin admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:09:44.091088 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9459bb4b-8fc5-4cd9-90cc-2f40486f4f74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:09:44.091753 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9459bb4b-8fc5-4cd9-90cc-2f40486f4f74 admin admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:09:44.092038 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9459bb4b-8fc5-4cd9-90cc-2f40486f4f74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:09:44.092319 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9459bb4b-8fc5-4cd9-90cc-2f40486f4f74 admin admin] PATH_INFO: `/v3/roles` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:09:44.092989 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9459bb4b-8fc5-4cd9-90cc-2f40486f4f74 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:09:44.094660 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9459bb4b-8fc5-4cd9-90cc-2f40486f4f74 admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:09:44.113105 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1029/2082] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 10 12:09:44 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 12:09:44.114155 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9459bb4b-8fc5-4cd9-90cc-2f40486f4f74 admin admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:09:44.126627 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312b1ad4-202c-4164-9271-1f0823afef41 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:09:44.127533 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9459bb4b-8fc5-4cd9-90cc-2f40486f4f74 admin admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:09:44.132793 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115f3a0e-0f40-465c-b283-22a3d78c7191 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:09:44.133482 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c814c28d-4fe1-4dc7-bde7-a2e7a1f56e32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:09:44.135914 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115f3a0e-0f40-465c-b283-22a3d78c7191 admin admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:09:44.135914 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115f3a0e-0f40-465c-b283-22a3d78c7191 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:09:44.135914 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115f3a0e-0f40-465c-b283-22a3d78c7191 admin admin] PATH_INFO: `/v3/roles` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:09:44.136169 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c814c28d-4fe1-4dc7-bde7-a2e7a1f56e32 admin admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:09:44.136169 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c814c28d-4fe1-4dc7-bde7-a2e7a1f56e32 admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:09:44.136325 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115f3a0e-0f40-465c-b283-22a3d78c7191 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:09:44.136396 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c814c28d-4fe1-4dc7-bde7-a2e7a1f56e32 admin admin] PATH_INFO: `/v3/roles` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:09:44.136396 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c814c28d-4fe1-4dc7-bde7-a2e7a1f56e32 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:09:44.137241 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c814c28d-4fe1-4dc7-bde7-a2e7a1f56e32 admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:09:44.137635 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115f3a0e-0f40-465c-b283-22a3d78c7191 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:09:44.151785 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1030/2083] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 10 12:09:44 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 12:09:44.151989 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1054/2084] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 10 12:09:44 2022] GET /identity/v3/roles => generated 1420 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 10 12:09:44.152543 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c814c28d-4fe1-4dc7-bde7-a2e7a1f56e32 admin admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:09:44.153078 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115f3a0e-0f40-465c-b283-22a3d78c7191 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:09:44.164040 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c814c28d-4fe1-4dc7-bde7-a2e7a1f56e32 admin admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:09:44.165353 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115f3a0e-0f40-465c-b283-22a3d78c7191 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:09:44.171722 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc37dfe-bd66-4261-bec3-49085b9ba886 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:09:44.177217 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc37dfe-bd66-4261-bec3-49085b9ba886 admin admin] REQUEST_METHOD: `PUT` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:09:44.177217 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc37dfe-bd66-4261-bec3-49085b9ba886 admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:09:44.177217 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc37dfe-bd66-4261-bec3-49085b9ba886 admin admin] PATH_INFO: `/v3/projects/3af5aaaaaeb54da8841d2ac03c5d5e13/users/7f2d05b3c1554b0fbc6bdbffa23eaf96/roles/0d822a0674b6495fac623877a3b38025` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:09:44.177927 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71b1754-97f7-4726-8c50-b970e4c86219 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:09:44.177927 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71b1754-97f7-4726-8c50-b970e4c86219 admin admin] REQUEST_METHOD: `PUT` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:09:44.177927 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71b1754-97f7-4726-8c50-b970e4c86219 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:09:44.177927 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71b1754-97f7-4726-8c50-b970e4c86219 admin admin] PATH_INFO: `/v3/projects/8e81b11a2b804a809894c8211c42d278/users/d93c36f2d1c1425d88c39e800c205066/roles/0d822a0674b6495fac623877a3b38025` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:09:44.178204 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:09:44.183699 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71b1754-97f7-4726-8c50-b970e4c86219 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8e81b11a2b804a809894c8211c42d278, user_id=d93c36f2d1c1425d88c39e800c205066, role_id=0d822a0674b6495fac623877a3b38025)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:09:44.189293 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71b1754-97f7-4726-8c50-b970e4c86219 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:09:44.191911 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc37dfe-bd66-4261-bec3-49085b9ba886 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3af5aaaaaeb54da8841d2ac03c5d5e13, user_id=7f2d05b3c1554b0fbc6bdbffa23eaf96, role_id=0d822a0674b6495fac623877a3b38025)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:09:44.194948 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc37dfe-bd66-4261-bec3-49085b9ba886 admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:09:44.218767 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1055/2085] 10.222.0.46 () {60 vars in 1483 bytes} [Mon Oct 10 12:09:44 2022] PUT /identity/v3/projects/8e81b11a2b804a809894c8211c42d278/users/d93c36f2d1c1425d88c39e800c205066/roles/0d822a0674b6495fac623877a3b38025 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:09:44.219855 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71b1754-97f7-4726-8c50-b970e4c86219 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:09:44.274911 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1031/2086] 10.222.0.46 () {60 vars in 1483 bytes} [Mon Oct 10 12:09:44 2022] PUT /identity/v3/projects/3af5aaaaaeb54da8841d2ac03c5d5e13/users/7f2d05b3c1554b0fbc6bdbffa23eaf96/roles/0d822a0674b6495fac623877a3b38025 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:09:44.274911 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfe805b-4d3d-4616-b8bd-154e02a39a5b None None] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:09:44.274911 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfe805b-4d3d-4616-b8bd-154e02a39a5b None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:09:44.274911 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfe805b-4d3d-4616-b8bd-154e02a39a5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:09:44.303305 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:09:44.309641 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.auth.core [None req-3dfe805b-4d3d-4616-b8bd-154e02a39a5b None None] MFA Rules not processed for user `d93c36f2d1c1425d88c39e800c205066`. Rule list: `[]` (Enabled: `True`). {{(pid=81676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:09:44.311487 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-3dfe805b-4d3d-4616-b8bd-154e02a39a5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:09:44.335262 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71b1754-97f7-4726-8c50-b970e4c86219 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:09:44.410592 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1032/2087] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:09:44 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:09:44.415726 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfe805b-4d3d-4616-b8bd-154e02a39a5b None None] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:09:44.428109 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f74d562-63ac-4d2e-8d47-8641cb669485 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:09:44.428865 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f74d562-63ac-4d2e-8d47-8641cb669485 admin admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:09:44.429345 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f74d562-63ac-4d2e-8d47-8641cb669485 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:09:44.429709 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f74d562-63ac-4d2e-8d47-8641cb669485 admin admin] PATH_INFO: `/v3/roles` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:09:44.430573 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f74d562-63ac-4d2e-8d47-8641cb669485 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:09:44.432379 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f74d562-63ac-4d2e-8d47-8641cb669485 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:09:44.443676 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1056/2088] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 10 12:09:44 2022] GET /identity/v3/roles => generated 1420 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 12:09:44.444556 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f74d562-63ac-4d2e-8d47-8641cb669485 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:09:44.450945 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-7f74d562-63ac-4d2e-8d47-8641cb669485 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:09:44.523659 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfe805b-4d3d-4616-b8bd-154e02a39a5b None None] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:09:44.529647 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96a3784-848d-4409-8aff-50d07cb80f72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:09:44.530272 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96a3784-848d-4409-8aff-50d07cb80f72 admin admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:09:44.530652 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96a3784-848d-4409-8aff-50d07cb80f72 admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:09:44.530860 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96a3784-848d-4409-8aff-50d07cb80f72 admin admin] PATH_INFO: `/v3/roles` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:09:44.531517 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96a3784-848d-4409-8aff-50d07cb80f72 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:09:44.533124 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96a3784-848d-4409-8aff-50d07cb80f72 admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:09:44.544511 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1033/2089] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 10 12:09:44 2022] GET /identity/v3/roles => generated 1420 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 12:09:44.545419 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96a3784-848d-4409-8aff-50d07cb80f72 admin admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:09:44.555735 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96a3784-848d-4409-8aff-50d07cb80f72 admin admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:09:44.557061 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f74d562-63ac-4d2e-8d47-8641cb669485 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:09:44.560936 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f221a5-c957-4d26-ba9d-23f810df1aa4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:09:44.561605 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f221a5-c957-4d26-ba9d-23f810df1aa4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:09:44.561892 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f221a5-c957-4d26-ba9d-23f810df1aa4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:09:44.562168 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f221a5-c957-4d26-ba9d-23f810df1aa4 admin admin] PATH_INFO: `/v3/projects/42363f6f6a6040e5b81070350a2b7855/users/13ac3580c0164d20994d68744d9153b8/roles/b2cb2a6e818e47338ce071e6fedfc673` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:09:44.563611 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ba74f5-588a-47cd-8d0a-d2c46c5e5d4b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3ae093470484b93b23dcfae1657df9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad3772f7b6a47c28b7ba37a9316a3c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:09:44.568222 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ba74f5-588a-47cd-8d0a-d2c46c5e5d4b service cinder] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:09:44.568222 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ba74f5-588a-47cd-8d0a-d2c46c5e5d4b service cinder] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:09:44.568222 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ba74f5-588a-47cd-8d0a-d2c46c5e5d4b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:09:44.569872 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-36ba74f5-588a-47cd-8d0a-d2c46c5e5d4b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:09:44.580707 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f221a5-c957-4d26-ba9d-23f810df1aa4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=42363f6f6a6040e5b81070350a2b7855, user_id=13ac3580c0164d20994d68744d9153b8, role_id=b2cb2a6e818e47338ce071e6fedfc673)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:09:44.584313 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f221a5-c957-4d26-ba9d-23f810df1aa4 admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:09:44.669956 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ba74f5-588a-47cd-8d0a-d2c46c5e5d4b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:09:44.672200 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ba74f5-588a-47cd-8d0a-d2c46c5e5d4b service cinder] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:09:44.679681 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1034/2090] 10.222.0.46 () {60 vars in 1483 bytes} [Mon Oct 10 12:09:44 2022] PUT /identity/v3/projects/42363f6f6a6040e5b81070350a2b7855/users/13ac3580c0164d20994d68744d9153b8/roles/b2cb2a6e818e47338ce071e6fedfc673 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:09:44.679681 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f221a5-c957-4d26-ba9d-23f810df1aa4 admin admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:09:44.688956 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1057/2091] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 10 12:09:44 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 245 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:09:44.691720 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0d57f0-8615-4663-8484-50b6320b8b9d None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:09:44.692174 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0d57f0-8615-4663-8484-50b6320b8b9d None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:09:44.692630 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0d57f0-8615-4663-8484-50b6320b8b9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:09:44.728732 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-8b0d57f0-8615-4663-8484-50b6320b8b9d None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:09:44.730859 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-8b0d57f0-8615-4663-8484-50b6320b8b9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:09:44.743482 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:09:44.775762 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f221a5-c957-4d26-ba9d-23f810df1aa4 admin admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:09:44.781463 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db97e156-8066-4325-b2fe-b52faa234b19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:09:44.782251 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db97e156-8066-4325-b2fe-b52faa234b19 admin admin] REQUEST_METHOD: `PUT` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:09:44.782535 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db97e156-8066-4325-b2fe-b52faa234b19 admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:09:44.782845 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db97e156-8066-4325-b2fe-b52faa234b19 admin admin] PATH_INFO: `/v3/projects/3af5aaaaaeb54da8841d2ac03c5d5e13/users/7f2d05b3c1554b0fbc6bdbffa23eaf96/roles/0d822a0674b6495fac623877a3b38025` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:09:44.786163 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db97e156-8066-4325-b2fe-b52faa234b19 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3af5aaaaaeb54da8841d2ac03c5d5e13, user_id=7f2d05b3c1554b0fbc6bdbffa23eaf96, role_id=0d822a0674b6495fac623877a3b38025)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:09:44.790570 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db97e156-8066-4325-b2fe-b52faa234b19 admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:09:44.803078 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:09:44.807981 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1035/2092] 10.222.0.46 () {60 vars in 1483 bytes} [Mon Oct 10 12:09:44 2022] PUT /identity/v3/projects/3af5aaaaaeb54da8841d2ac03c5d5e13/users/7f2d05b3c1554b0fbc6bdbffa23eaf96/roles/0d822a0674b6495fac623877a3b38025 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:09:44.821372 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dd3d31-5e7f-4e83-83fa-c7e057c8fc6a None None] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:09:44.821668 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dd3d31-5e7f-4e83-83fa-c7e057c8fc6a None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:09:44.822026 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dd3d31-5e7f-4e83-83fa-c7e057c8fc6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:09:44.857466 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1058/2093] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:09:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:09:44.865985 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.auth.core [None req-90dd3d31-5e7f-4e83-83fa-c7e057c8fc6a None None] MFA Rules not processed for user `7f2d05b3c1554b0fbc6bdbffa23eaf96`. Rule list: `[]` (Enabled: `True`). {{(pid=81676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:09:44.867848 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-90dd3d31-5e7f-4e83-83fa-c7e057c8fc6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:09:44.872254 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0d57f0-8615-4663-8484-50b6320b8b9d None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:09:44.874311 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-8b0d57f0-8615-4663-8484-50b6320b8b9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:09:44.982946 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0d57f0-8615-4663-8484-50b6320b8b9d None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:09:45.002517 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1036/2094] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:09:44 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:09:45.021333 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90dd3d31-5e7f-4e83-83fa-c7e057c8fc6a None None] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:09:45.023058 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-90dd3d31-5e7f-4e83-83fa-c7e057c8fc6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:09:45.057981 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45830653-786e-4533-85ef-3a6125695029 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58b1c7c85302421cba823ace4201861f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad3772f7b6a47c28b7ba37a9316a3c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:09:45.059285 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45830653-786e-4533-85ef-3a6125695029 service neutron] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:09:45.059844 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45830653-786e-4533-85ef-3a6125695029 service neutron] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:09:45.060314 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45830653-786e-4533-85ef-3a6125695029 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:09:45.115056 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90dd3d31-5e7f-4e83-83fa-c7e057c8fc6a None None] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:09:45.120183 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09d86c3-1e96-4345-8b82-90b43d030e42 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3ae093470484b93b23dcfae1657df9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad3772f7b6a47c28b7ba37a9316a3c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:09:45.120797 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09d86c3-1e96-4345-8b82-90b43d030e42 service cinder] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:09:45.121129 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09d86c3-1e96-4345-8b82-90b43d030e42 service cinder] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:09:45.121406 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09d86c3-1e96-4345-8b82-90b43d030e42 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:09:45.131999 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09d86c3-1e96-4345-8b82-90b43d030e42 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:09:45.133684 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09d86c3-1e96-4345-8b82-90b43d030e42 service cinder] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:09:45.145862 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1037/2095] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 10 12:09:45 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:09:45.150256 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45830653-786e-4533-85ef-3a6125695029 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:09:45.151723 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45830653-786e-4533-85ef-3a6125695029 service neutron] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:09:45.162305 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1059/2096] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 10 12:09:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 10 12:09:45.687737 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09d86c3-1e96-4345-8b82-90b43d030e42 service cinder] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:09:45.689178 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-d09d86c3-1e96-4345-8b82-90b43d030e42 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:09:45.726981 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09d86c3-1e96-4345-8b82-90b43d030e42 service cinder] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:09:45.735415 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739b422a-9413-4127-ad43-ac8fb41503ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:09:45.736142 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739b422a-9413-4127-ad43-ac8fb41503ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:09:45.742098 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739b422a-9413-4127-ad43-ac8fb41503ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:09:45.742630 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739b422a-9413-4127-ad43-ac8fb41503ea admin admin] PATH_INFO: `/v3/users/03d3aaf04d9d436b97618e4419c6366b` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:09:45.747571 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739b422a-9413-4127-ad43-ac8fb41503ea admin admin] RBAC: Authorizing `identity:delete_user(user_id=03d3aaf04d9d436b97618e4419c6366b)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:09:45.749424 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739b422a-9413-4127-ad43-ac8fb41503ea admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:09:46.148788 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-739b422a-9413-4127-ad43-ac8fb41503ea admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '03d3aaf04d9d436b97618e4419c6366b'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:09:46.163126 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-739b422a-9413-4127-ad43-ac8fb41503ea admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '03d3aaf04d9d436b97618e4419c6366b'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:09:46.164355 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-739b422a-9413-4127-ad43-ac8fb41503ea admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '03d3aaf04d9d436b97618e4419c6366b'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:09:46.169276 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:09:46.236346 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-739b422a-9413-4127-ad43-ac8fb41503ea admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '03d3aaf04d9d436b97618e4419c6366b'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:09:46.261860 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-739b422a-9413-4127-ad43-ac8fb41503ea admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '03d3aaf04d9d436b97618e4419c6366b'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:09:46.271359 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1038/2097] 10.222.0.46 () {58 vars in 1224 bytes} [Mon Oct 10 12:09:45 2022] DELETE /identity/v3/users/03d3aaf04d9d436b97618e4419c6366b => generated 0 bytes in 584 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:09:46.277660 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129a7b59-3d45-4c61-a976-6780b22383ed None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:09:46.283750 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129a7b59-3d45-4c61-a976-6780b22383ed None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:09:46.283750 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129a7b59-3d45-4c61-a976-6780b22383ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:09:46.301439 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-129a7b59-3d45-4c61-a976-6780b22383ed None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:09:46.302890 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-129a7b59-3d45-4c61-a976-6780b22383ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:09:46.363302 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:09:46.405673 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1060/2098] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:09:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 10 12:09:46.417736 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739b422a-9413-4127-ad43-ac8fb41503ea admin admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:09:46.420313 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-739b422a-9413-4127-ad43-ac8fb41503ea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:09:46.544800 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739b422a-9413-4127-ad43-ac8fb41503ea admin admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:09:46.550749 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861cddab-e694-4fef-bb0f-a53dd1a26570 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58b1c7c85302421cba823ace4201861f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad3772f7b6a47c28b7ba37a9316a3c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:09:46.552454 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861cddab-e694-4fef-bb0f-a53dd1a26570 service neutron] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:09:46.552454 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861cddab-e694-4fef-bb0f-a53dd1a26570 service neutron] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:09:46.552783 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861cddab-e694-4fef-bb0f-a53dd1a26570 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:09:46.576961 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129a7b59-3d45-4c61-a976-6780b22383ed None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:09:46.578720 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-129a7b59-3d45-4c61-a976-6780b22383ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:09:46.581094 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861cddab-e694-4fef-bb0f-a53dd1a26570 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:09:46.585599 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861cddab-e694-4fef-bb0f-a53dd1a26570 service neutron] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:09:46.601166 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129a7b59-3d45-4c61-a976-6780b22383ed None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:09:46.609127 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf8c422-8cb8-469a-a8e3-21b2d65c89d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:09:46.609815 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf8c422-8cb8-469a-a8e3-21b2d65c89d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:09:46.610249 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf8c422-8cb8-469a-a8e3-21b2d65c89d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:09:46.610605 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf8c422-8cb8-469a-a8e3-21b2d65c89d0 admin admin] PATH_INFO: `/v3/users/6c5eb0505d864d5f9c4caae177941f58` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:09:46.613275 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf8c422-8cb8-469a-a8e3-21b2d65c89d0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6c5eb0505d864d5f9c4caae177941f58)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:09:46.615616 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf8c422-8cb8-469a-a8e3-21b2d65c89d0 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:09:46.616170 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1039/2099] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 10 12:09:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:09:46.879242 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-fbf8c422-8cb8-469a-a8e3-21b2d65c89d0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6c5eb0505d864d5f9c4caae177941f58'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:09:46.886727 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-fbf8c422-8cb8-469a-a8e3-21b2d65c89d0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6c5eb0505d864d5f9c4caae177941f58'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:09:46.887450 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-fbf8c422-8cb8-469a-a8e3-21b2d65c89d0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6c5eb0505d864d5f9c4caae177941f58'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:09:47.017676 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-fbf8c422-8cb8-469a-a8e3-21b2d65c89d0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6c5eb0505d864d5f9c4caae177941f58'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:09:47.035348 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-fbf8c422-8cb8-469a-a8e3-21b2d65c89d0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6c5eb0505d864d5f9c4caae177941f58'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:09:47.044260 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1061/2100] 10.222.0.46 () {58 vars in 1224 bytes} [Mon Oct 10 12:09:46 2022] DELETE /identity/v3/users/6c5eb0505d864d5f9c4caae177941f58 => generated 0 bytes in 468 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:09:47.051762 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1d9ba3-c3b1-4e75-91f1-a8a4096aeea8 None None] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:09:47.052578 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1d9ba3-c3b1-4e75-91f1-a8a4096aeea8 None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:09:47.052578 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1d9ba3-c3b1-4e75-91f1-a8a4096aeea8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:09:47.090654 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.auth.core [None req-9d1d9ba3-c3b1-4e75-91f1-a8a4096aeea8 None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:09:47.093086 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-9d1d9ba3-c3b1-4e75-91f1-a8a4096aeea8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:09:47.197409 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4ab4da-1011-4f86-8f99-2cff5e24c92b None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:09:47.197409 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4ab4da-1011-4f86-8f99-2cff5e24c92b None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:09:47.197409 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4ab4da-1011-4f86-8f99-2cff5e24c92b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:09:47.197788 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1040/2101] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:09:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:09:47.204543 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1d9ba3-c3b1-4e75-91f1-a8a4096aeea8 None None] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:09:47.206225 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-9d1d9ba3-c3b1-4e75-91f1-a8a4096aeea8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:09:47.228934 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-6b4ab4da-1011-4f86-8f99-2cff5e24c92b None None] MFA Rules not processed for user `13ac3580c0164d20994d68744d9153b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:09:47.230493 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-6b4ab4da-1011-4f86-8f99-2cff5e24c92b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:09:47.294181 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1d9ba3-c3b1-4e75-91f1-a8a4096aeea8 None None] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:09:47.301162 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7500ad7f-4857-47e3-92ec-28e72435c6a2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58b1c7c85302421cba823ace4201861f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad3772f7b6a47c28b7ba37a9316a3c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:09:47.301993 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7500ad7f-4857-47e3-92ec-28e72435c6a2 service neutron] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:09:47.302359 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7500ad7f-4857-47e3-92ec-28e72435c6a2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:09:47.302729 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7500ad7f-4857-47e3-92ec-28e72435c6a2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:09:47.314715 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7500ad7f-4857-47e3-92ec-28e72435c6a2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:09:47.316324 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7500ad7f-4857-47e3-92ec-28e72435c6a2 service neutron] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:09:47.329113 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1062/2102] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:09:47 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 10 12:09:47.338420 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1041/2103] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 10 12:09:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 10 12:09:47.341879 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4ab4da-1011-4f86-8f99-2cff5e24c92b None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:09:47.342486 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7500ad7f-4857-47e3-92ec-28e72435c6a2 service neutron] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:09:47.343681 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-6b4ab4da-1011-4f86-8f99-2cff5e24c92b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:09:47.345147 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-7500ad7f-4857-47e3-92ec-28e72435c6a2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:09:47.365218 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4ab4da-1011-4f86-8f99-2cff5e24c92b None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:09:47.371111 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71634a6-abef-47b7-85df-8e2089026d28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:09:47.371806 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71634a6-abef-47b7-85df-8e2089026d28 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:09:47.372096 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71634a6-abef-47b7-85df-8e2089026d28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:09:47.372390 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71634a6-abef-47b7-85df-8e2089026d28 admin admin] PATH_INFO: `/v3/projects/8d6a8f944eb94b84bd67f0926ac637f8` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:09:47.373952 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71634a6-abef-47b7-85df-8e2089026d28 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8d6a8f944eb94b84bd67f0926ac637f8)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:09:47.376337 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71634a6-abef-47b7-85df-8e2089026d28 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:09:47.380738 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:DELETE=3,keystone:SELECT=97 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:09:47.424632 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7500ad7f-4857-47e3-92ec-28e72435c6a2 service neutron] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:09:47.429103 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9dc269-8d8b-4dc2-913b-a8bb417bbbe4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3ae093470484b93b23dcfae1657df9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad3772f7b6a47c28b7ba37a9316a3c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:09:47.429818 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9dc269-8d8b-4dc2-913b-a8bb417bbbe4 service cinder] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:09:47.430040 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9dc269-8d8b-4dc2-913b-a8bb417bbbe4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:09:47.430381 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9dc269-8d8b-4dc2-913b-a8bb417bbbe4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:09:47.439597 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9dc269-8d8b-4dc2-913b-a8bb417bbbe4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:09:47.441116 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9dc269-8d8b-4dc2-913b-a8bb417bbbe4 service cinder] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:09:47.452803 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1042/2104] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 10 12:09:47 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 10 12:09:47.495035 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:09:47.505367 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-f71634a6-abef-47b7-85df-8e2089026d28 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8d6a8f944eb94b84bd67f0926ac637f8', 'request_id': 'req-f71634a6-abef-47b7-85df-8e2089026d28'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:09:47.515361 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-f71634a6-abef-47b7-85df-8e2089026d28 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8d6a8f944eb94b84bd67f0926ac637f8', 'request_id': 'req-f71634a6-abef-47b7-85df-8e2089026d28'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:09:47.528675 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-f71634a6-abef-47b7-85df-8e2089026d28 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8d6a8f944eb94b84bd67f0926ac637f8', 'request_id': 'req-f71634a6-abef-47b7-85df-8e2089026d28'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:09:47.650314 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-f71634a6-abef-47b7-85df-8e2089026d28 admin admin] The token cache is being invalidate because project 8d6a8f944eb94b84bd67f0926ac637f8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81677) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 10 12:09:47.650575 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-f71634a6-abef-47b7-85df-8e2089026d28 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:09:47.651828 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-f71634a6-abef-47b7-85df-8e2089026d28 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:09:47.654599 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1063/2105] 10.222.0.46 () {58 vars in 1233 bytes} [Mon Oct 10 12:09:47 2022] DELETE /identity/v3/projects/8d6a8f944eb94b84bd67f0926ac637f8 => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:09:47.667674 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9dc269-8d8b-4dc2-913b-a8bb417bbbe4 service cinder] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:09:47.686819 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9dc269-8d8b-4dc2-913b-a8bb417bbbe4 service cinder] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:09:47.762242 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bd6614-ba46-4044-8c31-85c666250ca5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:09:47.762789 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bd6614-ba46-4044-8c31-85c666250ca5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:09:47.763204 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bd6614-ba46-4044-8c31-85c666250ca5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:09:47.763384 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bd6614-ba46-4044-8c31-85c666250ca5 admin admin] PATH_INFO: `/v3/users/2626b918b63b40d287fcf54a73b18be2` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:09:47.765431 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3bd6614-ba46-4044-8c31-85c666250ca5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2626b918b63b40d287fcf54a73b18be2)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:09:47.766757 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3bd6614-ba46-4044-8c31-85c666250ca5 admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:09:47.818843 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71634a6-abef-47b7-85df-8e2089026d28 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:09:47.820925 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-f71634a6-abef-47b7-85df-8e2089026d28 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:09:47.866974 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71634a6-abef-47b7-85df-8e2089026d28 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:09:47.879914 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96659892-9fb5-4ad0-9da1-1275b80d5cb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:09:47.881249 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96659892-9fb5-4ad0-9da1-1275b80d5cb3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:09:47.881505 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96659892-9fb5-4ad0-9da1-1275b80d5cb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:09:47.881995 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96659892-9fb5-4ad0-9da1-1275b80d5cb3 admin admin] PATH_INFO: `/v3/projects/fc3245b0042447f58d7297d1b414e400` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:09:47.884737 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96659892-9fb5-4ad0-9da1-1275b80d5cb3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fc3245b0042447f58d7297d1b414e400)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:09:47.889110 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96659892-9fb5-4ad0-9da1-1275b80d5cb3 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:09:48.028962 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-b3bd6614-ba46-4044-8c31-85c666250ca5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2626b918b63b40d287fcf54a73b18be2'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:09:48.039160 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-b3bd6614-ba46-4044-8c31-85c666250ca5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2626b918b63b40d287fcf54a73b18be2'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:09:48.042081 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-b3bd6614-ba46-4044-8c31-85c666250ca5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2626b918b63b40d287fcf54a73b18be2'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:09:48.063539 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-96659892-9fb5-4ad0-9da1-1275b80d5cb3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fc3245b0042447f58d7297d1b414e400', 'request_id': 'req-96659892-9fb5-4ad0-9da1-1275b80d5cb3'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:09:48.069950 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-96659892-9fb5-4ad0-9da1-1275b80d5cb3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fc3245b0042447f58d7297d1b414e400', 'request_id': 'req-96659892-9fb5-4ad0-9da1-1275b80d5cb3'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:09:48.077775 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-96659892-9fb5-4ad0-9da1-1275b80d5cb3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fc3245b0042447f58d7297d1b414e400', 'request_id': 'req-96659892-9fb5-4ad0-9da1-1275b80d5cb3'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:09:48.153514 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-b3bd6614-ba46-4044-8c31-85c666250ca5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2626b918b63b40d287fcf54a73b18be2'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:09:48.169882 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-b3bd6614-ba46-4044-8c31-85c666250ca5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2626b918b63b40d287fcf54a73b18be2'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:09:48.176826 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1043/2106] 10.222.0.46 () {58 vars in 1224 bytes} [Mon Oct 10 12:09:47 2022] DELETE /identity/v3/users/2626b918b63b40d287fcf54a73b18be2 => generated 0 bytes in 510 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:09:48.268378 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bd6614-ba46-4044-8c31-85c666250ca5 admin admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:09:48.270814 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-b3bd6614-ba46-4044-8c31-85c666250ca5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:09:48.271906 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-96659892-9fb5-4ad0-9da1-1275b80d5cb3 admin admin] The token cache is being invalidate because project fc3245b0042447f58d7297d1b414e400 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81677) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 10 12:09:48.274276 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-96659892-9fb5-4ad0-9da1-1275b80d5cb3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:09:48.274870 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-96659892-9fb5-4ad0-9da1-1275b80d5cb3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:09:48.277825 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1064/2107] 10.222.0.46 () {58 vars in 1233 bytes} [Mon Oct 10 12:09:47 2022] DELETE /identity/v3/projects/fc3245b0042447f58d7297d1b414e400 => generated 0 bytes in 460 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 10 12:09:48.283650 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96659892-9fb5-4ad0-9da1-1275b80d5cb3 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:09:48.286172 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-96659892-9fb5-4ad0-9da1-1275b80d5cb3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:09:48.348561 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:09:48.363663 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bd6614-ba46-4044-8c31-85c666250ca5 admin admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:09:48.369876 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce78f017-1e40-4760-aacd-7e1c2bfbcfd8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3ae093470484b93b23dcfae1657df9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad3772f7b6a47c28b7ba37a9316a3c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:09:48.370659 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce78f017-1e40-4760-aacd-7e1c2bfbcfd8 service cinder] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:09:48.370988 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce78f017-1e40-4760-aacd-7e1c2bfbcfd8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:09:48.371370 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce78f017-1e40-4760-aacd-7e1c2bfbcfd8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:09:48.373229 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-ce78f017-1e40-4760-aacd-7e1c2bfbcfd8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:09:48.384761 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96659892-9fb5-4ad0-9da1-1275b80d5cb3 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:09:48.495496 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db303621-2b04-4e7d-b2eb-cad25168309a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:09:48.496473 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db303621-2b04-4e7d-b2eb-cad25168309a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:09:48.496844 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db303621-2b04-4e7d-b2eb-cad25168309a admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:09:48.497317 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db303621-2b04-4e7d-b2eb-cad25168309a admin admin] PATH_INFO: `/v3/users/7f2d05b3c1554b0fbc6bdbffa23eaf96` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:09:48.501141 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db303621-2b04-4e7d-b2eb-cad25168309a admin admin] RBAC: Authorizing `identity:delete_user(user_id=7f2d05b3c1554b0fbc6bdbffa23eaf96)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:09:48.506168 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db303621-2b04-4e7d-b2eb-cad25168309a admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:09:48.514640 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce78f017-1e40-4760-aacd-7e1c2bfbcfd8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:09:48.518202 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce78f017-1e40-4760-aacd-7e1c2bfbcfd8 service cinder] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:09:48.537866 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1044/2108] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 10 12:09:48 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 270 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:09:48.718537 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce78f017-1e40-4760-aacd-7e1c2bfbcfd8 service cinder] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:09:48.720726 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-ce78f017-1e40-4760-aacd-7e1c2bfbcfd8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:09:48.751565 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce78f017-1e40-4760-aacd-7e1c2bfbcfd8 service cinder] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:09:48.760629 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-db303621-2b04-4e7d-b2eb-cad25168309a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7f2d05b3c1554b0fbc6bdbffa23eaf96'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:09:48.762121 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642bb3d1-a9ae-480e-8a2a-b426912fce21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:09:48.763387 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642bb3d1-a9ae-480e-8a2a-b426912fce21 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:09:48.763857 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642bb3d1-a9ae-480e-8a2a-b426912fce21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:09:48.764328 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642bb3d1-a9ae-480e-8a2a-b426912fce21 admin admin] PATH_INFO: `/v3/projects/b8932e30ceda436896269ff0e8aae42e` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:09:48.766444 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642bb3d1-a9ae-480e-8a2a-b426912fce21 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b8932e30ceda436896269ff0e8aae42e)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:09:48.774098 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642bb3d1-a9ae-480e-8a2a-b426912fce21 admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:09:48.779656 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-db303621-2b04-4e7d-b2eb-cad25168309a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7f2d05b3c1554b0fbc6bdbffa23eaf96'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:09:48.779656 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-db303621-2b04-4e7d-b2eb-cad25168309a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7f2d05b3c1554b0fbc6bdbffa23eaf96'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:09:48.850286 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-db303621-2b04-4e7d-b2eb-cad25168309a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7f2d05b3c1554b0fbc6bdbffa23eaf96'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:09:48.866189 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-db303621-2b04-4e7d-b2eb-cad25168309a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7f2d05b3c1554b0fbc6bdbffa23eaf96'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:09:48.872910 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1065/2109] 10.222.0.46 () {58 vars in 1224 bytes} [Mon Oct 10 12:09:48 2022] DELETE /identity/v3/users/7f2d05b3c1554b0fbc6bdbffa23eaf96 => generated 0 bytes in 590 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 10 12:09:48.951992 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-642bb3d1-a9ae-480e-8a2a-b426912fce21 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b8932e30ceda436896269ff0e8aae42e', 'request_id': 'req-642bb3d1-a9ae-480e-8a2a-b426912fce21'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:09:48.958496 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-642bb3d1-a9ae-480e-8a2a-b426912fce21 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b8932e30ceda436896269ff0e8aae42e', 'request_id': 'req-642bb3d1-a9ae-480e-8a2a-b426912fce21'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:09:48.966827 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-642bb3d1-a9ae-480e-8a2a-b426912fce21 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b8932e30ceda436896269ff0e8aae42e', 'request_id': 'req-642bb3d1-a9ae-480e-8a2a-b426912fce21'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:09:49.127010 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-642bb3d1-a9ae-480e-8a2a-b426912fce21 admin admin] The token cache is being invalidate because project b8932e30ceda436896269ff0e8aae42e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81676) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 10 12:09:49.127312 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-642bb3d1-a9ae-480e-8a2a-b426912fce21 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:09:49.128040 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-642bb3d1-a9ae-480e-8a2a-b426912fce21 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:09:49.133803 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1045/2110] 10.222.0.46 () {58 vars in 1233 bytes} [Mon Oct 10 12:09:48 2022] DELETE /identity/v3/projects/b8932e30ceda436896269ff0e8aae42e => generated 0 bytes in 413 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 10 12:09:49.134149 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db303621-2b04-4e7d-b2eb-cad25168309a admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:09:49.135808 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-db303621-2b04-4e7d-b2eb-cad25168309a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:09:49.160217 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:09:49.218510 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db303621-2b04-4e7d-b2eb-cad25168309a admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:09:49.314102 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ba65d2-a8f5-44c2-8a77-4123ab9476d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:09:49.314785 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ba65d2-a8f5-44c2-8a77-4123ab9476d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:09:49.315063 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ba65d2-a8f5-44c2-8a77-4123ab9476d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:09:49.315468 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ba65d2-a8f5-44c2-8a77-4123ab9476d4 admin admin] PATH_INFO: `/v3/users/420d942055b34777bfbca8d9f684c3e3` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:09:49.316516 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ba65d2-a8f5-44c2-8a77-4123ab9476d4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=420d942055b34777bfbca8d9f684c3e3)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:09:49.318458 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ba65d2-a8f5-44c2-8a77-4123ab9476d4 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:09:49.503270 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642bb3d1-a9ae-480e-8a2a-b426912fce21 admin admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:09:49.504783 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-642bb3d1-a9ae-480e-8a2a-b426912fce21 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:09:49.522801 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642bb3d1-a9ae-480e-8a2a-b426912fce21 admin admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:09:49.535205 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56018c88-e360-4b34-9512-6e6672e40612 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:09:49.535205 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56018c88-e360-4b34-9512-6e6672e40612 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:09:49.535205 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56018c88-e360-4b34-9512-6e6672e40612 admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:09:49.535205 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56018c88-e360-4b34-9512-6e6672e40612 admin admin] PATH_INFO: `/v3/projects/3af5aaaaaeb54da8841d2ac03c5d5e13` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:09:49.536336 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56018c88-e360-4b34-9512-6e6672e40612 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3af5aaaaaeb54da8841d2ac03c5d5e13)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:09:49.536336 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56018c88-e360-4b34-9512-6e6672e40612 admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:09:49.578455 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-01ba65d2-a8f5-44c2-8a77-4123ab9476d4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '420d942055b34777bfbca8d9f684c3e3'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:09:49.587477 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-01ba65d2-a8f5-44c2-8a77-4123ab9476d4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '420d942055b34777bfbca8d9f684c3e3'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:09:49.588590 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-01ba65d2-a8f5-44c2-8a77-4123ab9476d4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '420d942055b34777bfbca8d9f684c3e3'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:09:49.703449 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-01ba65d2-a8f5-44c2-8a77-4123ab9476d4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '420d942055b34777bfbca8d9f684c3e3'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:09:49.721764 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-01ba65d2-a8f5-44c2-8a77-4123ab9476d4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '420d942055b34777bfbca8d9f684c3e3'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:09:49.728125 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-56018c88-e360-4b34-9512-6e6672e40612 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3af5aaaaaeb54da8841d2ac03c5d5e13', 'request_id': 'req-56018c88-e360-4b34-9512-6e6672e40612'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:09:49.731688 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1066/2111] 10.222.0.46 () {58 vars in 1224 bytes} [Mon Oct 10 12:09:49 2022] DELETE /identity/v3/users/420d942055b34777bfbca8d9f684c3e3 => generated 0 bytes in 598 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 10 12:09:49.737467 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-56018c88-e360-4b34-9512-6e6672e40612 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3af5aaaaaeb54da8841d2ac03c5d5e13', 'request_id': 'req-56018c88-e360-4b34-9512-6e6672e40612'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:09:49.746460 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-56018c88-e360-4b34-9512-6e6672e40612 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3af5aaaaaeb54da8841d2ac03c5d5e13', 'request_id': 'req-56018c88-e360-4b34-9512-6e6672e40612'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:09:50.003909 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-56018c88-e360-4b34-9512-6e6672e40612 admin admin] The token cache is being invalidate because project 3af5aaaaaeb54da8841d2ac03c5d5e13 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81676) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 10 12:09:50.004432 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-56018c88-e360-4b34-9512-6e6672e40612 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:09:50.005706 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-56018c88-e360-4b34-9512-6e6672e40612 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:09:50.013849 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1046/2112] 10.222.0.46 () {58 vars in 1233 bytes} [Mon Oct 10 12:09:49 2022] DELETE /identity/v3/projects/3af5aaaaaeb54da8841d2ac03c5d5e13 => generated 0 bytes in 506 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 10 12:09:50.016681 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ff3862-c463-4fc7-9da4-b1dc97e9c394 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:09:50.017127 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ff3862-c463-4fc7-9da4-b1dc97e9c394 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:09:50.022826 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ff3862-c463-4fc7-9da4-b1dc97e9c394 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:09:50.045297 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-53ff3862-c463-4fc7-9da4-b1dc97e9c394 None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:09:50.047322 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-53ff3862-c463-4fc7-9da4-b1dc97e9c394 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:09:50.150282 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1067/2113] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:09:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:09:50.153459 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f44c6ab-2706-474b-87f1-2fe547082a0e None None] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:09:50.153961 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f44c6ab-2706-474b-87f1-2fe547082a0e None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:09:50.154461 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f44c6ab-2706-474b-87f1-2fe547082a0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:09:50.188450 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.auth.core [None req-4f44c6ab-2706-474b-87f1-2fe547082a0e None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:09:50.191101 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-4f44c6ab-2706-474b-87f1-2fe547082a0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:09:50.205197 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1047/2114] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:09:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:09:50.210442 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ff3862-c463-4fc7-9da4-b1dc97e9c394 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:09:50.229011 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ff3862-c463-4fc7-9da4-b1dc97e9c394 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:09:50.238129 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603a8f67-f291-4dd8-b131-082da6078602 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:09:50.239241 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603a8f67-f291-4dd8-b131-082da6078602 admin admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:09:50.239734 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603a8f67-f291-4dd8-b131-082da6078602 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:09:50.240213 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603a8f67-f291-4dd8-b131-082da6078602 admin admin] PATH_INFO: `/v3/domains` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:09:50.241423 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603a8f67-f291-4dd8-b131-082da6078602 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 12:09:50.241967 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603a8f67-f291-4dd8-b131-082da6078602 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:09:50.245126 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603a8f67-f291-4dd8-b131-082da6078602 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:09:50.267298 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1068/2115] 10.222.0.46 () {58 vars in 1153 bytes} [Mon Oct 10 12:09:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 12:09:50.273600 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f44c6ab-2706-474b-87f1-2fe547082a0e None None] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:09:50.287242 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:09:50.293436 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f44c6ab-2706-474b-87f1-2fe547082a0e None None] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:09:50.301191 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73f1bdf-6ecf-4bb9-83aa-7547f8424698 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:09:50.302277 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73f1bdf-6ecf-4bb9-83aa-7547f8424698 admin admin] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:09:50.302885 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73f1bdf-6ecf-4bb9-83aa-7547f8424698 admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:09:50.304038 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73f1bdf-6ecf-4bb9-83aa-7547f8424698 admin admin] PATH_INFO: `/v3/projects` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:09:50.304217 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73f1bdf-6ecf-4bb9-83aa-7547f8424698 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:09:50.308048 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73f1bdf-6ecf-4bb9-83aa-7547f8424698 admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:09:50.443025 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1048/2116] 10.222.0.46 () {60 vars in 1153 bytes} [Mon Oct 10 12:09:50 2022] POST /identity/v3/projects => generated 355 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 12:09:50.444851 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603a8f67-f291-4dd8-b131-082da6078602 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:09:50.478048 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73f1bdf-6ecf-4bb9-83aa-7547f8424698 admin admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:09:50.479883 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603a8f67-f291-4dd8-b131-082da6078602 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:09:50.488940 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-b73f1bdf-6ecf-4bb9-83aa-7547f8424698 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:09:50.634974 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89448fbd-81c7-45ed-8144-c972df331735 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:09:50.635831 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89448fbd-81c7-45ed-8144-c972df331735 admin admin] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:09:50.636073 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89448fbd-81c7-45ed-8144-c972df331735 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:09:50.636414 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89448fbd-81c7-45ed-8144-c972df331735 admin admin] PATH_INFO: `/v3/users` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:09:50.637148 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89448fbd-81c7-45ed-8144-c972df331735 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:09:50.639450 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89448fbd-81c7-45ed-8144-c972df331735 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:09:50.647866 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73f1bdf-6ecf-4bb9-83aa-7547f8424698 admin admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:09:50.656524 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a674b6a-d989-4449-89cf-49d5cdef165b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:09:50.657311 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a674b6a-d989-4449-89cf-49d5cdef165b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:09:50.657639 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a674b6a-d989-4449-89cf-49d5cdef165b admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:09:50.657956 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a674b6a-d989-4449-89cf-49d5cdef165b admin admin] PATH_INFO: `/v3/projects/b00f6a2a946f4af6a5506e5b483aa2b9` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:09:50.659865 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a674b6a-d989-4449-89cf-49d5cdef165b admin admin] RBAC: Authorizing `identity:delete_project(project_id=b00f6a2a946f4af6a5506e5b483aa2b9)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:09:50.662020 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a674b6a-d989-4449-89cf-49d5cdef165b admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:09:50.750573 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1069/2117] 10.222.0.46 () {60 vars in 1144 bytes} [Mon Oct 10 12:09:50 2022] POST /identity/v3/users => generated 278 bytes in 306 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 10 12:09:50.752160 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fced8e-1882-4664-94f2-3aeb79d6ecbd None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:09:50.752448 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fced8e-1882-4664-94f2-3aeb79d6ecbd None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:09:50.752732 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fced8e-1882-4664-94f2-3aeb79d6ecbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:09:50.774138 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:09:50.777000 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-a3fced8e-1882-4664-94f2-3aeb79d6ecbd None None] MFA Rules not processed for user `13ac3580c0164d20994d68744d9153b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:09:50.779020 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-a3fced8e-1882-4664-94f2-3aeb79d6ecbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:09:50.798493 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-9a674b6a-d989-4449-89cf-49d5cdef165b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b00f6a2a946f4af6a5506e5b483aa2b9', 'request_id': 'req-9a674b6a-d989-4449-89cf-49d5cdef165b'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:09:50.806075 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-9a674b6a-d989-4449-89cf-49d5cdef165b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b00f6a2a946f4af6a5506e5b483aa2b9', 'request_id': 'req-9a674b6a-d989-4449-89cf-49d5cdef165b'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:09:50.814199 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-9a674b6a-d989-4449-89cf-49d5cdef165b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b00f6a2a946f4af6a5506e5b483aa2b9', 'request_id': 'req-9a674b6a-d989-4449-89cf-49d5cdef165b'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:09:50.850623 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1070/2118] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:09:50 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:09:50.851799 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fced8e-1882-4664-94f2-3aeb79d6ecbd None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:09:50.853164 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-a3fced8e-1882-4664-94f2-3aeb79d6ecbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:09:50.923055 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-9a674b6a-d989-4449-89cf-49d5cdef165b admin admin] The token cache is being invalidate because project b00f6a2a946f4af6a5506e5b483aa2b9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81676) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 10 12:09:50.923708 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-9a674b6a-d989-4449-89cf-49d5cdef165b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:09:50.925551 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-9a674b6a-d989-4449-89cf-49d5cdef165b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:09:50.928854 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1049/2119] 10.222.0.46 () {58 vars in 1233 bytes} [Mon Oct 10 12:09:50 2022] DELETE /identity/v3/projects/b00f6a2a946f4af6a5506e5b483aa2b9 => generated 0 bytes in 451 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 10 12:09:50.930635 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a674b6a-d989-4449-89cf-49d5cdef165b admin admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:09:50.932627 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fced8e-1882-4664-94f2-3aeb79d6ecbd None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:09:50.934778 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-9a674b6a-d989-4449-89cf-49d5cdef165b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:09:50.937019 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49642b49-5193-48fb-9648-bf3521928f28 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3ae093470484b93b23dcfae1657df9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad3772f7b6a47c28b7ba37a9316a3c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:09:50.937646 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49642b49-5193-48fb-9648-bf3521928f28 service cinder] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:09:50.937962 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49642b49-5193-48fb-9648-bf3521928f28 service cinder] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:09:50.938359 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49642b49-5193-48fb-9648-bf3521928f28 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:09:50.941160 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-49642b49-5193-48fb-9648-bf3521928f28 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:09:51.025317 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49642b49-5193-48fb-9648-bf3521928f28 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:09:51.026817 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49642b49-5193-48fb-9648-bf3521928f28 service cinder] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:09:51.033115 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a674b6a-d989-4449-89cf-49d5cdef165b admin admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:09:51.039008 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ee9ca9-92e0-4866-a988-8a0c8c955bcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:09:51.039655 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ee9ca9-92e0-4866-a988-8a0c8c955bcb admin admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:09:51.039799 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1071/2120] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 10 12:09:50 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:09:51.039938 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ee9ca9-92e0-4866-a988-8a0c8c955bcb admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:09:51.040227 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ee9ca9-92e0-4866-a988-8a0c8c955bcb admin admin] PATH_INFO: `/v3/roles` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:09:51.040897 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ee9ca9-92e0-4866-a988-8a0c8c955bcb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:09:51.042032 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc95ea12-ad42-4244-bd7d-80b99c8604d5 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:09:51.042324 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc95ea12-ad42-4244-bd7d-80b99c8604d5 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:09:51.042592 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc95ea12-ad42-4244-bd7d-80b99c8604d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:09:51.042932 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ee9ca9-92e0-4866-a988-8a0c8c955bcb admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:09:51.056358 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1050/2121] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 10 12:09:50 2022] GET /identity/v3/roles => generated 1420 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 12:09:51.060600 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ee9ca9-92e0-4866-a988-8a0c8c955bcb admin admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:09:51.069721 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-fc95ea12-ad42-4244-bd7d-80b99c8604d5 None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:09:51.071182 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-fc95ea12-ad42-4244-bd7d-80b99c8604d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:09:51.072746 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ee9ca9-92e0-4866-a988-8a0c8c955bcb admin admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:09:51.078406 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fddb63-1bbb-497e-8b7a-b026c7d81351 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:09:51.078871 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fddb63-1bbb-497e-8b7a-b026c7d81351 admin admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:09:51.079158 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fddb63-1bbb-497e-8b7a-b026c7d81351 admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:09:51.079438 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fddb63-1bbb-497e-8b7a-b026c7d81351 admin admin] PATH_INFO: `/v3/roles` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:09:51.080372 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30fddb63-1bbb-497e-8b7a-b026c7d81351 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:09:51.081192 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1072/2122] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:09:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:09:51.082037 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30fddb63-1bbb-497e-8b7a-b026c7d81351 admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:09:51.095722 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a154d2-93d5-46f6-bcfc-324df805f7ae None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:09:51.096282 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a154d2-93d5-46f6-bcfc-324df805f7ae None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:09:51.097371 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a154d2-93d5-46f6-bcfc-324df805f7ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:09:51.097875 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1051/2123] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 10 12:09:51 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 10 12:09:51.098849 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fddb63-1bbb-497e-8b7a-b026c7d81351 admin admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:09:51.110711 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fddb63-1bbb-497e-8b7a-b026c7d81351 admin admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:09:51.116460 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e50ab2-22fb-42c3-8e45-765746236f98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:09:51.117166 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e50ab2-22fb-42c3-8e45-765746236f98 admin admin] REQUEST_METHOD: `PUT` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:09:51.117452 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e50ab2-22fb-42c3-8e45-765746236f98 admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:09:51.118118 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e50ab2-22fb-42c3-8e45-765746236f98 admin admin] PATH_INFO: `/v3/projects/a5a018e894f04195be6665f7281f5dbc/users/0c40870b57a443499f0926db4d2cf91a/roles/b2cb2a6e818e47338ce071e6fedfc673` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:09:51.142593 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-05a154d2-93d5-46f6-bcfc-324df805f7ae None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:09:51.144110 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-05a154d2-93d5-46f6-bcfc-324df805f7ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:09:51.149064 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e50ab2-22fb-42c3-8e45-765746236f98 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a5a018e894f04195be6665f7281f5dbc, user_id=0c40870b57a443499f0926db4d2cf91a, role_id=b2cb2a6e818e47338ce071e6fedfc673)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:09:51.152363 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e50ab2-22fb-42c3-8e45-765746236f98 admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:09:51.155724 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1073/2124] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:09:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:09:51.161016 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a154d2-93d5-46f6-bcfc-324df805f7ae None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:09:51.170915 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a154d2-93d5-46f6-bcfc-324df805f7ae None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:09:51.176906 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df211f1-b90b-4aaf-8778-ad27653cc2ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:09:51.177545 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df211f1-b90b-4aaf-8778-ad27653cc2ef admin admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:09:51.177826 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df211f1-b90b-4aaf-8778-ad27653cc2ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:09:51.178103 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df211f1-b90b-4aaf-8778-ad27653cc2ef admin admin] PATH_INFO: `/v3/domains` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:09:51.178847 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df211f1-b90b-4aaf-8778-ad27653cc2ef admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 12:09:51.179155 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df211f1-b90b-4aaf-8778-ad27653cc2ef admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:09:51.180782 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df211f1-b90b-4aaf-8778-ad27653cc2ef admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:09:51.194145 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1074/2125] 10.222.0.46 () {58 vars in 1153 bytes} [Mon Oct 10 12:09:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 12:09:51.199512 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df211f1-b90b-4aaf-8778-ad27653cc2ef admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:09:51.210575 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df211f1-b90b-4aaf-8778-ad27653cc2ef admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:09:51.215823 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab41d78-0aa2-4ab1-ac84-63bce21077c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:09:51.216462 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab41d78-0aa2-4ab1-ac84-63bce21077c0 admin admin] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:09:51.216745 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab41d78-0aa2-4ab1-ac84-63bce21077c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:09:51.217022 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab41d78-0aa2-4ab1-ac84-63bce21077c0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:09:51.217686 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab41d78-0aa2-4ab1-ac84-63bce21077c0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:09:51.219681 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab41d78-0aa2-4ab1-ac84-63bce21077c0 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:09:51.281354 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1052/2126] 10.222.0.46 () {60 vars in 1483 bytes} [Mon Oct 10 12:09:51 2022] PUT /identity/v3/projects/a5a018e894f04195be6665f7281f5dbc/users/0c40870b57a443499f0926db4d2cf91a/roles/b2cb2a6e818e47338ce071e6fedfc673 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:09:51.293805 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a75da5-0322-4c82-9739-580c0221e5c1 None None] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:09:51.294173 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a75da5-0322-4c82-9739-580c0221e5c1 None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:09:51.294421 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a75da5-0322-4c82-9739-580c0221e5c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:09:51.325270 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.auth.core [None req-a2a75da5-0322-4c82-9739-580c0221e5c1 None None] MFA Rules not processed for user `0c40870b57a443499f0926db4d2cf91a`. Rule list: `[]` (Enabled: `True`). {{(pid=81676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:09:51.326616 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-a2a75da5-0322-4c82-9739-580c0221e5c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:09:51.340124 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1075/2127] 10.222.0.46 () {60 vars in 1153 bytes} [Mon Oct 10 12:09:51 2022] POST /identity/v3/projects => generated 373 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 10 12:09:51.344384 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab41d78-0aa2-4ab1-ac84-63bce21077c0 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:09:51.417858 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1053/2128] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:09:51 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 10 12:09:51.421098 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a75da5-0322-4c82-9739-580c0221e5c1 None None] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:09:51.437263 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab41d78-0aa2-4ab1-ac84-63bce21077c0 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:09:51.453428 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:09:51.453607 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce948910-3504-4e5b-8258-190bec658d79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:09:51.453607 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce948910-3504-4e5b-8258-190bec658d79 admin admin] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:09:51.453607 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce948910-3504-4e5b-8258-190bec658d79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:09:51.453607 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce948910-3504-4e5b-8258-190bec658d79 admin admin] PATH_INFO: `/v3/users` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:09:51.453902 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce948910-3504-4e5b-8258-190bec658d79 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:09:51.453902 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce948910-3504-4e5b-8258-190bec658d79 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:09:51.515085 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a75da5-0322-4c82-9739-580c0221e5c1 None None] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:09:51.520753 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cec9da3-661e-410c-929f-0cb3fcdec074 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:09:51.521387 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cec9da3-661e-410c-929f-0cb3fcdec074 admin admin] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:09:51.522932 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cec9da3-661e-410c-929f-0cb3fcdec074 admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:09:51.523391 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cec9da3-661e-410c-929f-0cb3fcdec074 admin admin] PATH_INFO: `/v3/projects` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:09:51.524765 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cec9da3-661e-410c-929f-0cb3fcdec074 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:09:51.527993 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cec9da3-661e-410c-929f-0cb3fcdec074 admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:09:51.560740 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1076/2129] 10.222.0.46 () {60 vars in 1144 bytes} [Mon Oct 10 12:09:51 2022] POST /identity/v3/users => generated 287 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 10 12:09:51.565735 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce948910-3504-4e5b-8258-190bec658d79 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:09:51.583738 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce948910-3504-4e5b-8258-190bec658d79 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:09:51.591845 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49234ecf-8cf1-4e6d-b170-c46503f75ea4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:09:51.592908 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49234ecf-8cf1-4e6d-b170-c46503f75ea4 admin admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:09:51.593465 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49234ecf-8cf1-4e6d-b170-c46503f75ea4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:09:51.593957 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49234ecf-8cf1-4e6d-b170-c46503f75ea4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:09:51.595185 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49234ecf-8cf1-4e6d-b170-c46503f75ea4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:09:51.598949 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49234ecf-8cf1-4e6d-b170-c46503f75ea4 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:09:51.625453 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1077/2130] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 10 12:09:51 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 10 12:09:51.627945 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49234ecf-8cf1-4e6d-b170-c46503f75ea4 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:09:51.639488 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49234ecf-8cf1-4e6d-b170-c46503f75ea4 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:09:51.646637 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98ffc2f-9863-45bb-8188-8b6387981c2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:09:51.648439 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98ffc2f-9863-45bb-8188-8b6387981c2b admin admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:09:51.649713 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98ffc2f-9863-45bb-8188-8b6387981c2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:09:51.649713 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98ffc2f-9863-45bb-8188-8b6387981c2b admin admin] PATH_INFO: `/v3/roles` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:09:51.652048 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98ffc2f-9863-45bb-8188-8b6387981c2b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:09:51.655651 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98ffc2f-9863-45bb-8188-8b6387981c2b admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:09:51.674032 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1078/2131] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 10 12:09:51 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 10 12:09:51.679738 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98ffc2f-9863-45bb-8188-8b6387981c2b admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:09:51.695711 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98ffc2f-9863-45bb-8188-8b6387981c2b admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:09:51.703584 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca159940-4679-4836-b178-e493532fde4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:09:51.704651 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca159940-4679-4836-b178-e493532fde4d admin admin] REQUEST_METHOD: `PUT` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:09:51.705201 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca159940-4679-4836-b178-e493532fde4d admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:09:51.705670 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca159940-4679-4836-b178-e493532fde4d admin admin] PATH_INFO: `/v3/projects/4326aa9c66954652943d3754b121bce8/users/d58683f7689a4356badd8f4bb30850a5/roles/b2cb2a6e818e47338ce071e6fedfc673` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:09:51.709236 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1054/2132] 10.222.0.46 () {60 vars in 1153 bytes} [Mon Oct 10 12:09:51 2022] POST /identity/v3/projects => generated 355 bytes in 289 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 12:09:51.714431 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cec9da3-661e-410c-929f-0cb3fcdec074 admin admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:09:51.732617 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca159940-4679-4836-b178-e493532fde4d admin admin] RBAC: Authorizing `identity:create_grant(project_id=4326aa9c66954652943d3754b121bce8, user_id=d58683f7689a4356badd8f4bb30850a5, role_id=b2cb2a6e818e47338ce071e6fedfc673)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:09:51.738443 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca159940-4679-4836-b178-e493532fde4d admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:09:51.847836 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cec9da3-661e-410c-929f-0cb3fcdec074 admin admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:09:51.858820 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbfa0f6-d310-47ff-b601-e0ec64cd7e52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:09:51.859705 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbfa0f6-d310-47ff-b601-e0ec64cd7e52 admin admin] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:09:51.860154 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbfa0f6-d310-47ff-b601-e0ec64cd7e52 admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:09:51.860562 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbfa0f6-d310-47ff-b601-e0ec64cd7e52 admin admin] PATH_INFO: `/v3/users` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:09:51.861983 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfbfa0f6-d310-47ff-b601-e0ec64cd7e52 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:09:51.884191 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1079/2133] 10.222.0.46 () {60 vars in 1483 bytes} [Mon Oct 10 12:09:51 2022] PUT /identity/v3/projects/4326aa9c66954652943d3754b121bce8/users/d58683f7689a4356badd8f4bb30850a5/roles/b2cb2a6e818e47338ce071e6fedfc673 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:09:51.884820 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfbfa0f6-d310-47ff-b601-e0ec64cd7e52 admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:09:51.887229 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5628bc-2e41-449b-9e24-e7970b3a2854 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:09:51.888318 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5628bc-2e41-449b-9e24-e7970b3a2854 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:09:51.888318 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5628bc-2e41-449b-9e24-e7970b3a2854 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:09:51.932652 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-1c5628bc-2e41-449b-9e24-e7970b3a2854 None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:09:51.933780 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-1c5628bc-2e41-449b-9e24-e7970b3a2854 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:09:52.009367 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1055/2134] 10.222.0.46 () {60 vars in 1144 bytes} [Mon Oct 10 12:09:51 2022] POST /identity/v3/users => generated 284 bytes in 296 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 12:09:52.013974 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbfa0f6-d310-47ff-b601-e0ec64cd7e52 admin admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:09:52.026361 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbfa0f6-d310-47ff-b601-e0ec64cd7e52 admin admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:09:52.031864 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1080/2135] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:09:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 10 12:09:52.042800 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5628bc-2e41-449b-9e24-e7970b3a2854 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:09:52.044603 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e83cdb-b96e-4b68-9294-53b5cf0fc561 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:09:52.050841 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-1c5628bc-2e41-449b-9e24-e7970b3a2854 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:09:52.051005 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e83cdb-b96e-4b68-9294-53b5cf0fc561 admin admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:09:52.051005 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e83cdb-b96e-4b68-9294-53b5cf0fc561 admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:09:52.051005 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e83cdb-b96e-4b68-9294-53b5cf0fc561 admin admin] PATH_INFO: `/v3/roles` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:09:52.051005 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e83cdb-b96e-4b68-9294-53b5cf0fc561 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:09:52.051683 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e83cdb-b96e-4b68-9294-53b5cf0fc561 admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:09:52.069546 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1056/2136] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 10 12:09:52 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 10 12:09:52.073966 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e83cdb-b96e-4b68-9294-53b5cf0fc561 admin admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:09:52.085220 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e83cdb-b96e-4b68-9294-53b5cf0fc561 admin admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:09:52.090331 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27176d78-5ac7-4d83-b964-8feffc615165 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:09:52.091075 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27176d78-5ac7-4d83-b964-8feffc615165 admin admin] REQUEST_METHOD: `PUT` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:09:52.091350 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27176d78-5ac7-4d83-b964-8feffc615165 admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:09:52.091627 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27176d78-5ac7-4d83-b964-8feffc615165 admin admin] PATH_INFO: `/v3/projects/ad57ccf6538b449fb6a866f4322a027c/users/75daef2c3e0140e8bf941399a17d34e2/roles/0d822a0674b6495fac623877a3b38025` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:09:52.109694 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27176d78-5ac7-4d83-b964-8feffc615165 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ad57ccf6538b449fb6a866f4322a027c, user_id=75daef2c3e0140e8bf941399a17d34e2, role_id=0d822a0674b6495fac623877a3b38025)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:09:52.113145 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27176d78-5ac7-4d83-b964-8feffc615165 admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:09:52.128794 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5628bc-2e41-449b-9e24-e7970b3a2854 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:09:52.132742 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436a2517-324a-4548-a1b0-0acbcc48430c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58b1c7c85302421cba823ace4201861f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad3772f7b6a47c28b7ba37a9316a3c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:09:52.133370 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436a2517-324a-4548-a1b0-0acbcc48430c service neutron] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:09:52.133650 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436a2517-324a-4548-a1b0-0acbcc48430c service neutron] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:09:52.133919 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436a2517-324a-4548-a1b0-0acbcc48430c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:09:52.144693 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:09:52.154210 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436a2517-324a-4548-a1b0-0acbcc48430c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:09:52.154210 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436a2517-324a-4548-a1b0-0acbcc48430c service neutron] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:09:52.159943 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1081/2137] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 10 12:09:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:09:52.262050 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1057/2138] 10.222.0.46 () {60 vars in 1483 bytes} [Mon Oct 10 12:09:52 2022] PUT /identity/v3/projects/ad57ccf6538b449fb6a866f4322a027c/users/75daef2c3e0140e8bf941399a17d34e2/roles/0d822a0674b6495fac623877a3b38025 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 10 12:09:52.266192 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436a2517-324a-4548-a1b0-0acbcc48430c service neutron] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:09:52.276631 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436a2517-324a-4548-a1b0-0acbcc48430c service neutron] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:09:52.337317 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3ac7c6-1d27-4734-92a4-7a22a941107c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:09:52.338041 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3ac7c6-1d27-4734-92a4-7a22a941107c admin admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:09:52.338319 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3ac7c6-1d27-4734-92a4-7a22a941107c admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:09:52.338598 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3ac7c6-1d27-4734-92a4-7a22a941107c admin admin] PATH_INFO: `/v3/roles` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:09:52.339288 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3ac7c6-1d27-4734-92a4-7a22a941107c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:09:52.340936 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3ac7c6-1d27-4734-92a4-7a22a941107c admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:09:52.356322 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1082/2139] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 10 12:09:52 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 10 12:09:52.358004 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27176d78-5ac7-4d83-b964-8feffc615165 admin admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:09:52.371854 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27176d78-5ac7-4d83-b964-8feffc615165 admin admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:09:52.377461 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e419c1-de76-4871-8582-84f0e2845850 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:09:52.378201 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e419c1-de76-4871-8582-84f0e2845850 admin admin] REQUEST_METHOD: `PUT` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:09:52.378491 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e419c1-de76-4871-8582-84f0e2845850 admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:09:52.378961 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e419c1-de76-4871-8582-84f0e2845850 admin admin] PATH_INFO: `/v3/projects/ad57ccf6538b449fb6a866f4322a027c/users/75daef2c3e0140e8bf941399a17d34e2/roles/0d822a0674b6495fac623877a3b38025` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:09:52.381086 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e419c1-de76-4871-8582-84f0e2845850 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ad57ccf6538b449fb6a866f4322a027c, user_id=75daef2c3e0140e8bf941399a17d34e2, role_id=0d822a0674b6495fac623877a3b38025)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:09:52.383886 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e419c1-de76-4871-8582-84f0e2845850 admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:09:52.396120 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1058/2140] 10.222.0.46 () {60 vars in 1483 bytes} [Mon Oct 10 12:09:52 2022] PUT /identity/v3/projects/ad57ccf6538b449fb6a866f4322a027c/users/75daef2c3e0140e8bf941399a17d34e2/roles/0d822a0674b6495fac623877a3b38025 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:09:52.407025 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91833477-f492-4215-ae10-acb1e719c2ef None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:09:52.407307 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91833477-f492-4215-ae10-acb1e719c2ef None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:09:52.407705 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91833477-f492-4215-ae10-acb1e719c2ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:09:52.440100 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-91833477-f492-4215-ae10-acb1e719c2ef None None] MFA Rules not processed for user `75daef2c3e0140e8bf941399a17d34e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:09:52.441383 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-91833477-f492-4215-ae10-acb1e719c2ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:09:52.529852 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1083/2141] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:09:52 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:09:52.547790 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e419c1-de76-4871-8582-84f0e2845850 admin admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:09:52.552247 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-58e419c1-de76-4871-8582-84f0e2845850 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:09:52.674058 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e419c1-de76-4871-8582-84f0e2845850 admin admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:09:52.678805 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f2a4e9-3af2-42d5-bd8e-66de00339888 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3ae093470484b93b23dcfae1657df9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad3772f7b6a47c28b7ba37a9316a3c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:09:52.679594 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f2a4e9-3af2-42d5-bd8e-66de00339888 service cinder] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:09:52.680009 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f2a4e9-3af2-42d5-bd8e-66de00339888 service cinder] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:09:52.680393 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f2a4e9-3af2-42d5-bd8e-66de00339888 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:09:52.692174 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f2a4e9-3af2-42d5-bd8e-66de00339888 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:09:52.693636 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f2a4e9-3af2-42d5-bd8e-66de00339888 service cinder] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:09:52.705262 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1059/2142] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 10 12:09:52 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:09:53.025523 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91833477-f492-4215-ae10-acb1e719c2ef None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:09:53.051461 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91833477-f492-4215-ae10-acb1e719c2ef None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:09:53.065397 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2134fc-b008-4a48-a61b-81673cd16ee4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3ae093470484b93b23dcfae1657df9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad3772f7b6a47c28b7ba37a9316a3c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:09:53.065397 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2134fc-b008-4a48-a61b-81673cd16ee4 service cinder] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:09:53.066961 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2134fc-b008-4a48-a61b-81673cd16ee4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:09:53.067722 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2134fc-b008-4a48-a61b-81673cd16ee4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:09:53.123080 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:09:53.157246 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2134fc-b008-4a48-a61b-81673cd16ee4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:09:53.158659 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2134fc-b008-4a48-a61b-81673cd16ee4 service cinder] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:09:53.187333 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1084/2143] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 10 12:09:53 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:09:53.567323 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:09:53.920446 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5dc112-0c90-4454-8807-e36852851797 None None] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:09:53.920734 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5dc112-0c90-4454-8807-e36852851797 None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:09:53.921193 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5dc112-0c90-4454-8807-e36852851797 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:09:53.950271 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.auth.core [None req-2d5dc112-0c90-4454-8807-e36852851797 None None] MFA Rules not processed for user `d58683f7689a4356badd8f4bb30850a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:09:53.951707 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-2d5dc112-0c90-4454-8807-e36852851797 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:09:54.038942 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1060/2144] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:09:53 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:09:54.050926 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2134fc-b008-4a48-a61b-81673cd16ee4 service cinder] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:09:54.060999 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2134fc-b008-4a48-a61b-81673cd16ee4 service cinder] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:09:54.065521 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf43b4a-2c44-487c-9669-bf91840ab4aa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3ae093470484b93b23dcfae1657df9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad3772f7b6a47c28b7ba37a9316a3c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:09:54.066516 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf43b4a-2c44-487c-9669-bf91840ab4aa service cinder] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:09:54.067132 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf43b4a-2c44-487c-9669-bf91840ab4aa service cinder] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:09:54.067568 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf43b4a-2c44-487c-9669-bf91840ab4aa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:09:54.082568 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf43b4a-2c44-487c-9669-bf91840ab4aa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:09:54.085285 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf43b4a-2c44-487c-9669-bf91840ab4aa service cinder] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:09:54.099245 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1085/2145] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 10 12:09:54 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:09:54.244726 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:09:57.126653 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c86649d-c4f8-48b2-b832-d453ef59c76c None None] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:09:57.127230 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c86649d-c4f8-48b2-b832-d453ef59c76c None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:09:57.127365 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c86649d-c4f8-48b2-b832-d453ef59c76c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:09:57.147782 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.auth.core [None req-4c86649d-c4f8-48b2-b832-d453ef59c76c None None] MFA Rules not processed for user `d58683f7689a4356badd8f4bb30850a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:09:57.149020 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-4c86649d-c4f8-48b2-b832-d453ef59c76c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:09:57.156139 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1061/2146] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:09:57 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:09:57.609327 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf43b4a-2c44-487c-9669-bf91840ab4aa service cinder] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:09:57.613881 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-caf43b4a-2c44-487c-9669-bf91840ab4aa service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:09:57.664376 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:09:57.720174 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf43b4a-2c44-487c-9669-bf91840ab4aa service cinder] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:09:57.725426 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf214d29-e730-4ef9-8db4-3b475d5c36e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:09:57.726754 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf214d29-e730-4ef9-8db4-3b475d5c36e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:09:57.727214 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf214d29-e730-4ef9-8db4-3b475d5c36e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:09:57.727664 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf214d29-e730-4ef9-8db4-3b475d5c36e8 admin admin] PATH_INFO: `/v3/users/42d58ead76bd4dedaef6dea6014f1b22` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:09:57.728794 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf214d29-e730-4ef9-8db4-3b475d5c36e8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=42d58ead76bd4dedaef6dea6014f1b22)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:09:57.731592 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf214d29-e730-4ef9-8db4-3b475d5c36e8 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:09:57.993896 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-cf214d29-e730-4ef9-8db4-3b475d5c36e8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '42d58ead76bd4dedaef6dea6014f1b22'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:09:58.009484 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-cf214d29-e730-4ef9-8db4-3b475d5c36e8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '42d58ead76bd4dedaef6dea6014f1b22'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:09:58.010900 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-cf214d29-e730-4ef9-8db4-3b475d5c36e8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '42d58ead76bd4dedaef6dea6014f1b22'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:09:58.109569 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-cf214d29-e730-4ef9-8db4-3b475d5c36e8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '42d58ead76bd4dedaef6dea6014f1b22'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:09:58.148249 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-cf214d29-e730-4ef9-8db4-3b475d5c36e8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '42d58ead76bd4dedaef6dea6014f1b22'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:09:58.167033 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1086/2147] 10.222.0.46 () {58 vars in 1224 bytes} [Mon Oct 10 12:09:57 2022] DELETE /identity/v3/users/42d58ead76bd4dedaef6dea6014f1b22 => generated 0 bytes in 558 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:09:58.172615 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797116eb-a3ee-46ec-8da0-568a1a41c531 None None] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:09:58.173184 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797116eb-a3ee-46ec-8da0-568a1a41c531 None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:09:58.173554 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797116eb-a3ee-46ec-8da0-568a1a41c531 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:09:58.224797 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.auth.core [None req-797116eb-a3ee-46ec-8da0-568a1a41c531 None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:09:58.226450 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-797116eb-a3ee-46ec-8da0-568a1a41c531 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:09:58.231554 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:09:58.306898 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1062/2148] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:09:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:09:58.321622 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf214d29-e730-4ef9-8db4-3b475d5c36e8 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:09:58.324190 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-cf214d29-e730-4ef9-8db4-3b475d5c36e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:09:58.401474 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf214d29-e730-4ef9-8db4-3b475d5c36e8 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:09:58.408880 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d10b80b-3bba-4e4d-8653-a4f57cad901b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58b1c7c85302421cba823ace4201861f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad3772f7b6a47c28b7ba37a9316a3c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:09:58.409535 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d10b80b-3bba-4e4d-8653-a4f57cad901b service neutron] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:09:58.410343 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d10b80b-3bba-4e4d-8653-a4f57cad901b service neutron] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:09:58.410629 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d10b80b-3bba-4e4d-8653-a4f57cad901b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:09:58.428177 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d10b80b-3bba-4e4d-8653-a4f57cad901b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:09:58.432448 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d10b80b-3bba-4e4d-8653-a4f57cad901b service neutron] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:09:58.443172 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1087/2149] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 10 12:09:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:09:59.301415 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797116eb-a3ee-46ec-8da0-568a1a41c531 None None] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:09:59.303628 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-797116eb-a3ee-46ec-8da0-568a1a41c531 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:09:59.320714 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797116eb-a3ee-46ec-8da0-568a1a41c531 None None] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:09:59.326704 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d45b0a-df7d-4ba6-a5dd-1a48a9e72a00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:09:59.327722 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d45b0a-df7d-4ba6-a5dd-1a48a9e72a00 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:09:59.329129 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d45b0a-df7d-4ba6-a5dd-1a48a9e72a00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:09:59.329129 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d45b0a-df7d-4ba6-a5dd-1a48a9e72a00 admin admin] PATH_INFO: `/v3/projects/647106e6a44d4e2ea3a19aa4ca09f8fa` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:09:59.330104 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d45b0a-df7d-4ba6-a5dd-1a48a9e72a00 admin admin] RBAC: Authorizing `identity:delete_project(project_id=647106e6a44d4e2ea3a19aa4ca09f8fa)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:09:59.332223 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d45b0a-df7d-4ba6-a5dd-1a48a9e72a00 admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:09:59.517374 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-14d45b0a-df7d-4ba6-a5dd-1a48a9e72a00 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '647106e6a44d4e2ea3a19aa4ca09f8fa', 'request_id': 'req-14d45b0a-df7d-4ba6-a5dd-1a48a9e72a00'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:09:59.554844 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-14d45b0a-df7d-4ba6-a5dd-1a48a9e72a00 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '647106e6a44d4e2ea3a19aa4ca09f8fa', 'request_id': 'req-14d45b0a-df7d-4ba6-a5dd-1a48a9e72a00'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:09:59.561034 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-14d45b0a-df7d-4ba6-a5dd-1a48a9e72a00 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '647106e6a44d4e2ea3a19aa4ca09f8fa', 'request_id': 'req-14d45b0a-df7d-4ba6-a5dd-1a48a9e72a00'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:09:59.689062 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-14d45b0a-df7d-4ba6-a5dd-1a48a9e72a00 admin admin] The token cache is being invalidate because project 647106e6a44d4e2ea3a19aa4ca09f8fa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81676) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 10 12:09:59.690614 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-14d45b0a-df7d-4ba6-a5dd-1a48a9e72a00 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:09:59.691560 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-14d45b0a-df7d-4ba6-a5dd-1a48a9e72a00 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:09:59.693383 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1063/2150] 10.222.0.46 () {58 vars in 1233 bytes} [Mon Oct 10 12:09:59 2022] DELETE /identity/v3/projects/647106e6a44d4e2ea3a19aa4ca09f8fa => generated 0 bytes in 394 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:09:59.699852 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d10b80b-3bba-4e4d-8653-a4f57cad901b service neutron] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:09:59.727048 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d10b80b-3bba-4e4d-8653-a4f57cad901b service neutron] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:09:59.778220 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:09:59.797530 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42bd3f71-1127-4956-a20d-1f95f239e5dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:09:59.798644 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bd3f71-1127-4956-a20d-1f95f239e5dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:09:59.799190 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bd3f71-1127-4956-a20d-1f95f239e5dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:09:59.799938 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bd3f71-1127-4956-a20d-1f95f239e5dc admin admin] PATH_INFO: `/v3/users/b6714679417841a6a1a0f71d1af255d0` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:09:59.802023 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42bd3f71-1127-4956-a20d-1f95f239e5dc admin admin] RBAC: Authorizing `identity:delete_user(user_id=b6714679417841a6a1a0f71d1af255d0)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:09:59.805903 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42bd3f71-1127-4956-a20d-1f95f239e5dc admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:10:00.067133 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-42bd3f71-1127-4956-a20d-1f95f239e5dc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b6714679417841a6a1a0f71d1af255d0'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:10:00.103024 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-42bd3f71-1127-4956-a20d-1f95f239e5dc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b6714679417841a6a1a0f71d1af255d0'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:10:00.104816 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-42bd3f71-1127-4956-a20d-1f95f239e5dc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b6714679417841a6a1a0f71d1af255d0'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:10:00.205565 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-42bd3f71-1127-4956-a20d-1f95f239e5dc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b6714679417841a6a1a0f71d1af255d0'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:10:00.281542 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-42bd3f71-1127-4956-a20d-1f95f239e5dc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b6714679417841a6a1a0f71d1af255d0'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:10:00.293999 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1088/2151] 10.222.0.46 () {58 vars in 1224 bytes} [Mon Oct 10 12:09:59 2022] DELETE /identity/v3/users/b6714679417841a6a1a0f71d1af255d0 => generated 0 bytes in 593 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:10:00.866040 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d45b0a-df7d-4ba6-a5dd-1a48a9e72a00 admin admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:10:00.869013 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-14d45b0a-df7d-4ba6-a5dd-1a48a9e72a00 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:10:00.889032 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d45b0a-df7d-4ba6-a5dd-1a48a9e72a00 admin admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:10:00.974899 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02643ef1-26eb-408d-aa74-1de0db7d35ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:10:00.975695 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02643ef1-26eb-408d-aa74-1de0db7d35ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:10:00.976315 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02643ef1-26eb-408d-aa74-1de0db7d35ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:10:00.976733 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02643ef1-26eb-408d-aa74-1de0db7d35ae admin admin] PATH_INFO: `/v3/projects/6eab5e80b43d47119bbc61441a423ccd` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:10:00.978392 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02643ef1-26eb-408d-aa74-1de0db7d35ae admin admin] RBAC: Authorizing `identity:delete_project(project_id=6eab5e80b43d47119bbc61441a423ccd)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:10:00.980995 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02643ef1-26eb-408d-aa74-1de0db7d35ae admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:10:01.120252 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:10:01.134701 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-02643ef1-26eb-408d-aa74-1de0db7d35ae admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6eab5e80b43d47119bbc61441a423ccd', 'request_id': 'req-02643ef1-26eb-408d-aa74-1de0db7d35ae'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:10:01.141716 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-02643ef1-26eb-408d-aa74-1de0db7d35ae admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6eab5e80b43d47119bbc61441a423ccd', 'request_id': 'req-02643ef1-26eb-408d-aa74-1de0db7d35ae'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:10:01.149433 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-02643ef1-26eb-408d-aa74-1de0db7d35ae admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6eab5e80b43d47119bbc61441a423ccd', 'request_id': 'req-02643ef1-26eb-408d-aa74-1de0db7d35ae'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:10:01.263993 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-02643ef1-26eb-408d-aa74-1de0db7d35ae admin admin] The token cache is being invalidate because project 6eab5e80b43d47119bbc61441a423ccd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81676) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 10 12:10:01.263993 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-02643ef1-26eb-408d-aa74-1de0db7d35ae admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:10:01.263993 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-02643ef1-26eb-408d-aa74-1de0db7d35ae admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:10:01.268672 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1064/2152] 10.222.0.46 () {58 vars in 1233 bytes} [Mon Oct 10 12:10:00 2022] DELETE /identity/v3/projects/6eab5e80b43d47119bbc61441a423ccd => generated 0 bytes in 403 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:10:01.272504 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42bd3f71-1127-4956-a20d-1f95f239e5dc admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:10:01.274040 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-42bd3f71-1127-4956-a20d-1f95f239e5dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:10:01.294713 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42bd3f71-1127-4956-a20d-1f95f239e5dc admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:10:01.418986 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804ba17f-97b2-4cb7-9f9b-9ba8df21345c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:10:01.419652 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804ba17f-97b2-4cb7-9f9b-9ba8df21345c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:10:01.419932 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804ba17f-97b2-4cb7-9f9b-9ba8df21345c admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:10:01.420918 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804ba17f-97b2-4cb7-9f9b-9ba8df21345c admin admin] PATH_INFO: `/v3/users/d93c36f2d1c1425d88c39e800c205066` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:10:01.421608 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804ba17f-97b2-4cb7-9f9b-9ba8df21345c admin admin] RBAC: Authorizing `identity:delete_user(user_id=d93c36f2d1c1425d88c39e800c205066)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:10:01.423982 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804ba17f-97b2-4cb7-9f9b-9ba8df21345c admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:10:01.695151 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-804ba17f-97b2-4cb7-9f9b-9ba8df21345c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd93c36f2d1c1425d88c39e800c205066'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:10:01.722996 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-804ba17f-97b2-4cb7-9f9b-9ba8df21345c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd93c36f2d1c1425d88c39e800c205066'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:10:01.724105 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-804ba17f-97b2-4cb7-9f9b-9ba8df21345c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd93c36f2d1c1425d88c39e800c205066'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:10:01.801474 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-804ba17f-97b2-4cb7-9f9b-9ba8df21345c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd93c36f2d1c1425d88c39e800c205066'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:10:01.805112 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=80,keystone:DELETE=18,keystone:INSERT=2 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:10:01.825577 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-804ba17f-97b2-4cb7-9f9b-9ba8df21345c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd93c36f2d1c1425d88c39e800c205066'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:10:01.832975 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1089/2153] 10.222.0.46 () {58 vars in 1224 bytes} [Mon Oct 10 12:10:01 2022] DELETE /identity/v3/users/d93c36f2d1c1425d88c39e800c205066 => generated 0 bytes in 561 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:10:02.435156 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02643ef1-26eb-408d-aa74-1de0db7d35ae admin admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:10:02.437526 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-02643ef1-26eb-408d-aa74-1de0db7d35ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:10:02.452317 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02643ef1-26eb-408d-aa74-1de0db7d35ae admin admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:10:02.552407 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3a4ef0-c68d-4a7d-b83e-6399d5a66065 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:10:02.553195 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3a4ef0-c68d-4a7d-b83e-6399d5a66065 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:10:02.553504 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3a4ef0-c68d-4a7d-b83e-6399d5a66065 admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:10:02.553783 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3a4ef0-c68d-4a7d-b83e-6399d5a66065 admin admin] PATH_INFO: `/v3/projects/8e81b11a2b804a809894c8211c42d278` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:10:02.555517 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3a4ef0-c68d-4a7d-b83e-6399d5a66065 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8e81b11a2b804a809894c8211c42d278)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:10:02.557518 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3a4ef0-c68d-4a7d-b83e-6399d5a66065 admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:10:02.739055 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-7c3a4ef0-c68d-4a7d-b83e-6399d5a66065 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8e81b11a2b804a809894c8211c42d278', 'request_id': 'req-7c3a4ef0-c68d-4a7d-b83e-6399d5a66065'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:10:02.756126 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-7c3a4ef0-c68d-4a7d-b83e-6399d5a66065 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8e81b11a2b804a809894c8211c42d278', 'request_id': 'req-7c3a4ef0-c68d-4a7d-b83e-6399d5a66065'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:10:02.771259 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-7c3a4ef0-c68d-4a7d-b83e-6399d5a66065 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8e81b11a2b804a809894c8211c42d278', 'request_id': 'req-7c3a4ef0-c68d-4a7d-b83e-6399d5a66065'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:10:02.915848 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-7c3a4ef0-c68d-4a7d-b83e-6399d5a66065 admin admin] The token cache is being invalidate because project 8e81b11a2b804a809894c8211c42d278 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81676) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 10 12:10:02.916151 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-7c3a4ef0-c68d-4a7d-b83e-6399d5a66065 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:10:02.916879 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-7c3a4ef0-c68d-4a7d-b83e-6399d5a66065 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:10:02.919134 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1065/2154] 10.222.0.46 () {58 vars in 1233 bytes} [Mon Oct 10 12:10:02 2022] DELETE /identity/v3/projects/8e81b11a2b804a809894c8211c42d278 => generated 0 bytes in 485 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:10:02.926872 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91897bb4-4355-4da9-9a9b-4853fed89b85 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:10:02.927656 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91897bb4-4355-4da9-9a9b-4853fed89b85 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:10:02.927656 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91897bb4-4355-4da9-9a9b-4853fed89b85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:10:02.956179 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-91897bb4-4355-4da9-9a9b-4853fed89b85 None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:10:02.957704 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-91897bb4-4355-4da9-9a9b-4853fed89b85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:10:03.081241 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1090/2155] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:10:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:10:03.092735 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa5ffff-9e2d-4da2-9f34-58ec5037cbf3 None None] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:10:03.094784 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa5ffff-9e2d-4da2-9f34-58ec5037cbf3 None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:10:03.095079 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa5ffff-9e2d-4da2-9f34-58ec5037cbf3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:10:03.128096 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.auth.core [None req-6fa5ffff-9e2d-4da2-9f34-58ec5037cbf3 None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:10:03.129405 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-6fa5ffff-9e2d-4da2-9f34-58ec5037cbf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:10:03.140384 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1066/2156] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:10:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:10:03.145200 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91897bb4-4355-4da9-9a9b-4853fed89b85 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:10:03.171272 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91897bb4-4355-4da9-9a9b-4853fed89b85 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:10:03.181497 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a9cfab-dac9-4d2d-b00b-65450ef13b66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:10:03.182539 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a9cfab-dac9-4d2d-b00b-65450ef13b66 admin admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:10:03.183085 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a9cfab-dac9-4d2d-b00b-65450ef13b66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:10:03.183631 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a9cfab-dac9-4d2d-b00b-65450ef13b66 admin admin] PATH_INFO: `/v3/domains` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:10:03.184899 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a9cfab-dac9-4d2d-b00b-65450ef13b66 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 12:10:03.185478 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a9cfab-dac9-4d2d-b00b-65450ef13b66 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:10:03.188783 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a9cfab-dac9-4d2d-b00b-65450ef13b66 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:10:03.210027 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1091/2157] 10.222.0.46 () {58 vars in 1153 bytes} [Mon Oct 10 12:10:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 12:10:03.216566 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa5ffff-9e2d-4da2-9f34-58ec5037cbf3 None None] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:10:03.228172 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa5ffff-9e2d-4da2-9f34-58ec5037cbf3 None None] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:10:03.233606 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f3f042-7824-4247-af81-742326b7b8f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:10:03.234277 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f3f042-7824-4247-af81-742326b7b8f7 admin admin] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:10:03.234537 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f3f042-7824-4247-af81-742326b7b8f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:10:03.234833 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f3f042-7824-4247-af81-742326b7b8f7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:10:03.235513 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f3f042-7824-4247-af81-742326b7b8f7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:10:03.238111 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f3f042-7824-4247-af81-742326b7b8f7 admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:10:03.360106 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1067/2158] 10.222.0.46 () {60 vars in 1153 bytes} [Mon Oct 10 12:10:03 2022] POST /identity/v3/projects => generated 377 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 10 12:10:03.365426 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a9cfab-dac9-4d2d-b00b-65450ef13b66 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:10:03.378303 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a9cfab-dac9-4d2d-b00b-65450ef13b66 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:10:03.443681 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8225ce3-d3d0-4cfa-9343-892afe0dcff1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:10:03.444603 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8225ce3-d3d0-4cfa-9343-892afe0dcff1 admin admin] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:10:03.445046 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8225ce3-d3d0-4cfa-9343-892afe0dcff1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:10:03.445346 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8225ce3-d3d0-4cfa-9343-892afe0dcff1 admin admin] PATH_INFO: `/v3/users` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:10:03.446023 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8225ce3-d3d0-4cfa-9343-892afe0dcff1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:10:03.448095 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8225ce3-d3d0-4cfa-9343-892afe0dcff1 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:10:03.539017 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1092/2159] 10.222.0.46 () {60 vars in 1144 bytes} [Mon Oct 10 12:10:03 2022] POST /identity/v3/users => generated 289 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 12:10:03.543537 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f3f042-7824-4247-af81-742326b7b8f7 admin admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:10:03.554856 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f3f042-7824-4247-af81-742326b7b8f7 admin admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:10:03.560035 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72e49b7-fea8-47f8-b535-ba41d86d5102 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:10:03.560665 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72e49b7-fea8-47f8-b535-ba41d86d5102 admin admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:10:03.560946 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72e49b7-fea8-47f8-b535-ba41d86d5102 admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:10:03.561222 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72e49b7-fea8-47f8-b535-ba41d86d5102 admin admin] PATH_INFO: `/v3/roles` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:10:03.561888 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72e49b7-fea8-47f8-b535-ba41d86d5102 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:10:03.563605 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72e49b7-fea8-47f8-b535-ba41d86d5102 admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:10:03.576255 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1068/2160] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 10 12:10:03 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 12:10:03.581681 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8225ce3-d3d0-4cfa-9343-892afe0dcff1 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:10:03.593329 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8225ce3-d3d0-4cfa-9343-892afe0dcff1 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:10:03.599463 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5a981c-37ed-40f4-911e-7bbafb5c7508 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:10:03.600027 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5a981c-37ed-40f4-911e-7bbafb5c7508 admin admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:10:03.600310 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5a981c-37ed-40f4-911e-7bbafb5c7508 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:10:03.600588 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5a981c-37ed-40f4-911e-7bbafb5c7508 admin admin] PATH_INFO: `/v3/roles` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:10:03.601504 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5a981c-37ed-40f4-911e-7bbafb5c7508 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:10:03.603196 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5a981c-37ed-40f4-911e-7bbafb5c7508 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:10:03.615136 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1093/2161] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 10 12:10:03 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 12:10:03.619510 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72e49b7-fea8-47f8-b535-ba41d86d5102 admin admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:10:03.630244 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72e49b7-fea8-47f8-b535-ba41d86d5102 admin admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:10:03.635628 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d573b3a8-a31d-448a-a37a-89abf2016a9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:10:03.636312 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d573b3a8-a31d-448a-a37a-89abf2016a9d admin admin] REQUEST_METHOD: `PUT` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:10:03.636596 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d573b3a8-a31d-448a-a37a-89abf2016a9d admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:10:03.636875 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d573b3a8-a31d-448a-a37a-89abf2016a9d admin admin] PATH_INFO: `/v3/projects/26773079b6914a20ab27109b0dec496a/users/21c537b0fefe4fb1b51483ac0e3cf2d1/roles/b2cb2a6e818e47338ce071e6fedfc673` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:10:03.650230 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:DELETE=12,keystone:SELECT=85,keystone:INSERT=3 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:10:03.654638 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d573b3a8-a31d-448a-a37a-89abf2016a9d admin admin] RBAC: Authorizing `identity:create_grant(project_id=26773079b6914a20ab27109b0dec496a, user_id=21c537b0fefe4fb1b51483ac0e3cf2d1, role_id=b2cb2a6e818e47338ce071e6fedfc673)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:10:03.657418 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d573b3a8-a31d-448a-a37a-89abf2016a9d admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:10:03.778283 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1069/2162] 10.222.0.46 () {60 vars in 1483 bytes} [Mon Oct 10 12:10:03 2022] PUT /identity/v3/projects/26773079b6914a20ab27109b0dec496a/users/21c537b0fefe4fb1b51483ac0e3cf2d1/roles/b2cb2a6e818e47338ce071e6fedfc673 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:10:03.785958 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da1267d-0a94-4ec8-a4d8-04d8704b03fb None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:10:03.786245 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da1267d-0a94-4ec8-a4d8-04d8704b03fb None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:10:03.786592 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da1267d-0a94-4ec8-a4d8-04d8704b03fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:10:03.808774 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-8da1267d-0a94-4ec8-a4d8-04d8704b03fb None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:10:03.810026 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-8da1267d-0a94-4ec8-a4d8-04d8704b03fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:10:03.865208 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:10:03.882650 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1094/2163] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:10:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:10:03.893937 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d573b3a8-a31d-448a-a37a-89abf2016a9d admin admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:10:03.896336 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-d573b3a8-a31d-448a-a37a-89abf2016a9d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:10:03.959032 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d573b3a8-a31d-448a-a37a-89abf2016a9d admin admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:10:03.962925 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdf7155-e6da-449b-8c11-b290c2ba8624 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58b1c7c85302421cba823ace4201861f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad3772f7b6a47c28b7ba37a9316a3c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:10:03.963549 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdf7155-e6da-449b-8c11-b290c2ba8624 service neutron] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:10:03.963827 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdf7155-e6da-449b-8c11-b290c2ba8624 service neutron] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:10:03.964110 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdf7155-e6da-449b-8c11-b290c2ba8624 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:10:03.975100 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efdf7155-e6da-449b-8c11-b290c2ba8624 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:10:03.976894 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efdf7155-e6da-449b-8c11-b290c2ba8624 service neutron] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:10:03.999044 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1070/2164] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 10 12:10:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:10:10.842445 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37a98b5-631e-42dd-8d2a-c12cb0c26b4f None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:10:10.843910 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37a98b5-631e-42dd-8d2a-c12cb0c26b4f None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:10:10.844584 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37a98b5-631e-42dd-8d2a-c12cb0c26b4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:10:10.887052 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-e37a98b5-631e-42dd-8d2a-c12cb0c26b4f None None] MFA Rules not processed for user `45c87aa85b7d45fe8acf6f36a3d2a639`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:10:10.890042 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-e37a98b5-631e-42dd-8d2a-c12cb0c26b4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:10:11.016353 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1095/2165] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:10:10 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:10:14.375413 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933e32ee-b0da-4e3b-ab3b-c980eccee480 None None] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:10:14.375962 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933e32ee-b0da-4e3b-ab3b-c980eccee480 None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:10:14.376125 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933e32ee-b0da-4e3b-ab3b-c980eccee480 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:10:14.411464 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.auth.core [None req-933e32ee-b0da-4e3b-ab3b-c980eccee480 None None] MFA Rules not processed for user `21c537b0fefe4fb1b51483ac0e3cf2d1`. Rule list: `[]` (Enabled: `True`). {{(pid=81676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:10:14.412693 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-933e32ee-b0da-4e3b-ab3b-c980eccee480 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:10:14.510992 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1071/2166] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:10:14 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 10 12:10:14.516010 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37a98b5-631e-42dd-8d2a-c12cb0c26b4f None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:10:14.530955 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37a98b5-631e-42dd-8d2a-c12cb0c26b4f None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:10:14.537064 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9194c578-e350-42d2-aedb-75fc091699ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:10:14.537713 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9194c578-e350-42d2-aedb-75fc091699ff admin admin] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:10:14.538088 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9194c578-e350-42d2-aedb-75fc091699ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:10:14.538246 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9194c578-e350-42d2-aedb-75fc091699ff admin admin] PATH_INFO: `/v3/projects` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:10:14.539489 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9194c578-e350-42d2-aedb-75fc091699ff admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:10:14.541930 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9194c578-e350-42d2-aedb-75fc091699ff admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:10:14.696423 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1096/2167] 10.222.0.46 () {60 vars in 1153 bytes} [Mon Oct 10 12:10:14 2022] POST /identity/v3/projects => generated 377 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 12:10:14.697649 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933e32ee-b0da-4e3b-ab3b-c980eccee480 None None] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:10:14.723212 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933e32ee-b0da-4e3b-ab3b-c980eccee480 None None] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:10:14.832584 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816f9fa7-58f1-4fd8-9d68-899eb4ed4b4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:10:14.833637 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816f9fa7-58f1-4fd8-9d68-899eb4ed4b4b admin admin] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:10:14.834645 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816f9fa7-58f1-4fd8-9d68-899eb4ed4b4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:10:14.834645 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816f9fa7-58f1-4fd8-9d68-899eb4ed4b4b admin admin] PATH_INFO: `/v3/users` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:10:14.835791 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816f9fa7-58f1-4fd8-9d68-899eb4ed4b4b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:10:14.839981 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816f9fa7-58f1-4fd8-9d68-899eb4ed4b4b admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:10:14.939364 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1072/2168] 10.222.0.46 () {60 vars in 1144 bytes} [Mon Oct 10 12:10:14 2022] POST /identity/v3/users => generated 295 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 12:10:14.943389 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9194c578-e350-42d2-aedb-75fc091699ff admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:10:14.959312 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9194c578-e350-42d2-aedb-75fc091699ff admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:10:14.966231 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac18b2f1-0663-4c33-855d-5aab69506b70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:10:14.967569 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac18b2f1-0663-4c33-855d-5aab69506b70 admin admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:10:14.968069 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac18b2f1-0663-4c33-855d-5aab69506b70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:10:14.968493 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac18b2f1-0663-4c33-855d-5aab69506b70 admin admin] PATH_INFO: `/v3/roles` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:10:14.969321 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac18b2f1-0663-4c33-855d-5aab69506b70 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:10:14.971739 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac18b2f1-0663-4c33-855d-5aab69506b70 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:10:14.984730 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1097/2169] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 10 12:10:14 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 12:10:14.990992 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816f9fa7-58f1-4fd8-9d68-899eb4ed4b4b admin admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:10:15.006170 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816f9fa7-58f1-4fd8-9d68-899eb4ed4b4b admin admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:10:15.013536 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e514b8e0-3987-48cd-a35e-5539613a160b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:10:15.014298 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e514b8e0-3987-48cd-a35e-5539613a160b admin admin] REQUEST_METHOD: `PUT` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:10:15.014587 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e514b8e0-3987-48cd-a35e-5539613a160b admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:10:15.014922 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e514b8e0-3987-48cd-a35e-5539613a160b admin admin] PATH_INFO: `/v3/projects/568ccc02e08d4bc6bccfe42c58fcf97e/users/09668960625845f0ad79bb91a6de7766/roles/0d822a0674b6495fac623877a3b38025` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:10:15.028607 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:10:15.038771 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e514b8e0-3987-48cd-a35e-5539613a160b admin admin] RBAC: Authorizing `identity:create_grant(project_id=568ccc02e08d4bc6bccfe42c58fcf97e, user_id=09668960625845f0ad79bb91a6de7766, role_id=0d822a0674b6495fac623877a3b38025)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:10:15.043844 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e514b8e0-3987-48cd-a35e-5539613a160b admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:10:15.128683 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1073/2170] 10.222.0.46 () {60 vars in 1483 bytes} [Mon Oct 10 12:10:14 2022] PUT /identity/v3/projects/568ccc02e08d4bc6bccfe42c58fcf97e/users/09668960625845f0ad79bb91a6de7766/roles/0d822a0674b6495fac623877a3b38025 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:10:15.133143 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac18b2f1-0663-4c33-855d-5aab69506b70 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:10:15.151473 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac18b2f1-0663-4c33-855d-5aab69506b70 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:10:15.288386 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417e4814-7338-4f01-ad2e-174fb71e1cf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:10:15.289911 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417e4814-7338-4f01-ad2e-174fb71e1cf0 admin admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:10:15.291121 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417e4814-7338-4f01-ad2e-174fb71e1cf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:10:15.292144 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417e4814-7338-4f01-ad2e-174fb71e1cf0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:10:15.293956 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417e4814-7338-4f01-ad2e-174fb71e1cf0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:10:15.301420 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417e4814-7338-4f01-ad2e-174fb71e1cf0 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:10:15.320723 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1098/2171] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 10 12:10:15 2022] GET /identity/v3/roles => generated 1420 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 10 12:10:15.326832 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e514b8e0-3987-48cd-a35e-5539613a160b admin admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:10:15.344730 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e514b8e0-3987-48cd-a35e-5539613a160b admin admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:10:15.353931 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90950792-ece8-41b2-abd8-fe2efe704246 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:10:15.355359 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90950792-ece8-41b2-abd8-fe2efe704246 admin admin] REQUEST_METHOD: `PUT` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:10:15.356045 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90950792-ece8-41b2-abd8-fe2efe704246 admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:10:15.356667 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90950792-ece8-41b2-abd8-fe2efe704246 admin admin] PATH_INFO: `/v3/projects/568ccc02e08d4bc6bccfe42c58fcf97e/users/09668960625845f0ad79bb91a6de7766/roles/0d822a0674b6495fac623877a3b38025` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:10:15.361492 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90950792-ece8-41b2-abd8-fe2efe704246 admin admin] RBAC: Authorizing `identity:create_grant(project_id=568ccc02e08d4bc6bccfe42c58fcf97e, user_id=09668960625845f0ad79bb91a6de7766, role_id=0d822a0674b6495fac623877a3b38025)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:10:15.367764 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90950792-ece8-41b2-abd8-fe2efe704246 admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:10:15.399054 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1074/2172] 10.222.0.46 () {60 vars in 1483 bytes} [Mon Oct 10 12:10:15 2022] PUT /identity/v3/projects/568ccc02e08d4bc6bccfe42c58fcf97e/users/09668960625845f0ad79bb91a6de7766/roles/0d822a0674b6495fac623877a3b38025 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:10:21.312510 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417e4814-7338-4f01-ad2e-174fb71e1cf0 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:10:21.315446 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-417e4814-7338-4f01-ad2e-174fb71e1cf0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:10:21.335364 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417e4814-7338-4f01-ad2e-174fb71e1cf0 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:10:21.421214 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:10:21.429835 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950cb621-1175-4cfc-a02a-c7e241968d3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:10:21.431414 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950cb621-1175-4cfc-a02a-c7e241968d3f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:10:21.432118 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950cb621-1175-4cfc-a02a-c7e241968d3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:10:21.432879 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950cb621-1175-4cfc-a02a-c7e241968d3f admin admin] PATH_INFO: `/v3/users/0c40870b57a443499f0926db4d2cf91a` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:10:21.435012 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950cb621-1175-4cfc-a02a-c7e241968d3f admin admin] RBAC: Authorizing `identity:delete_user(user_id=0c40870b57a443499f0926db4d2cf91a)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:10:21.438864 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950cb621-1175-4cfc-a02a-c7e241968d3f admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:10:21.809093 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-950cb621-1175-4cfc-a02a-c7e241968d3f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0c40870b57a443499f0926db4d2cf91a'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:10:21.826418 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-950cb621-1175-4cfc-a02a-c7e241968d3f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0c40870b57a443499f0926db4d2cf91a'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:10:21.832281 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-950cb621-1175-4cfc-a02a-c7e241968d3f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0c40870b57a443499f0926db4d2cf91a'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:10:21.950037 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-950cb621-1175-4cfc-a02a-c7e241968d3f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0c40870b57a443499f0926db4d2cf91a'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:10:22.002066 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-950cb621-1175-4cfc-a02a-c7e241968d3f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0c40870b57a443499f0926db4d2cf91a'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:10:22.026814 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1099/2173] 10.222.0.46 () {58 vars in 1224 bytes} [Mon Oct 10 12:10:21 2022] DELETE /identity/v3/users/0c40870b57a443499f0926db4d2cf91a => generated 0 bytes in 715 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:10:22.033704 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c43b30-52cd-44d7-a7f6-cba8fefb2733 None None] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:10:22.034795 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c43b30-52cd-44d7-a7f6-cba8fefb2733 None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:10:22.035281 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c43b30-52cd-44d7-a7f6-cba8fefb2733 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:10:22.067991 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.auth.core [None req-c9c43b30-52cd-44d7-a7f6-cba8fefb2733 None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:10:22.069640 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-c9c43b30-52cd-44d7-a7f6-cba8fefb2733 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:10:22.256700 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1075/2174] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:10:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:10:22.278224 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950cb621-1175-4cfc-a02a-c7e241968d3f admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:10:22.280314 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-950cb621-1175-4cfc-a02a-c7e241968d3f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:10:22.433348 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950cb621-1175-4cfc-a02a-c7e241968d3f admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:10:22.439119 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6221c697-fd32-482b-9a83-f13fcc530335 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58b1c7c85302421cba823ace4201861f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad3772f7b6a47c28b7ba37a9316a3c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:10:22.440487 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6221c697-fd32-482b-9a83-f13fcc530335 service neutron] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:10:22.440786 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6221c697-fd32-482b-9a83-f13fcc530335 service neutron] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:10:22.441080 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6221c697-fd32-482b-9a83-f13fcc530335 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:10:22.459485 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6221c697-fd32-482b-9a83-f13fcc530335 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:10:22.461693 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6221c697-fd32-482b-9a83-f13fcc530335 service neutron] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:10:22.478957 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1100/2175] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 10 12:10:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:10:23.614234 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c43b30-52cd-44d7-a7f6-cba8fefb2733 None None] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:10:23.618133 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-c9c43b30-52cd-44d7-a7f6-cba8fefb2733 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:10:23.652701 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c43b30-52cd-44d7-a7f6-cba8fefb2733 None None] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:10:23.662398 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fcfdf6-0374-4511-8f62-59e1206e7e18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:10:23.664614 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fcfdf6-0374-4511-8f62-59e1206e7e18 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:10:23.672725 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fcfdf6-0374-4511-8f62-59e1206e7e18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:10:23.672725 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fcfdf6-0374-4511-8f62-59e1206e7e18 admin admin] PATH_INFO: `/v3/projects/a5a018e894f04195be6665f7281f5dbc` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:10:23.672725 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5fcfdf6-0374-4511-8f62-59e1206e7e18 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a5a018e894f04195be6665f7281f5dbc)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:10:23.672725 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5fcfdf6-0374-4511-8f62-59e1206e7e18 admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:10:23.892121 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-e5fcfdf6-0374-4511-8f62-59e1206e7e18 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a5a018e894f04195be6665f7281f5dbc', 'request_id': 'req-e5fcfdf6-0374-4511-8f62-59e1206e7e18'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:10:23.923326 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-e5fcfdf6-0374-4511-8f62-59e1206e7e18 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a5a018e894f04195be6665f7281f5dbc', 'request_id': 'req-e5fcfdf6-0374-4511-8f62-59e1206e7e18'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:10:23.939221 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-e5fcfdf6-0374-4511-8f62-59e1206e7e18 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a5a018e894f04195be6665f7281f5dbc', 'request_id': 'req-e5fcfdf6-0374-4511-8f62-59e1206e7e18'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:10:24.023477 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-e5fcfdf6-0374-4511-8f62-59e1206e7e18 admin admin] The token cache is being invalidate because project a5a018e894f04195be6665f7281f5dbc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81676) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 10 12:10:24.024417 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-e5fcfdf6-0374-4511-8f62-59e1206e7e18 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:10:24.026050 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-e5fcfdf6-0374-4511-8f62-59e1206e7e18 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:10:24.029273 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1076/2176] 10.222.0.46 () {58 vars in 1233 bytes} [Mon Oct 10 12:10:23 2022] DELETE /identity/v3/projects/a5a018e894f04195be6665f7281f5dbc => generated 0 bytes in 418 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:10:24.037208 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6221c697-fd32-482b-9a83-f13fcc530335 service neutron] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:10:24.057807 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6221c697-fd32-482b-9a83-f13fcc530335 service neutron] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:10:24.188764 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcc06c4-1147-4f49-890a-dee941692df9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:10:24.191124 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcc06c4-1147-4f49-890a-dee941692df9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:10:24.191934 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcc06c4-1147-4f49-890a-dee941692df9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:10:24.197527 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcc06c4-1147-4f49-890a-dee941692df9 admin admin] PATH_INFO: `/v3/users/75daef2c3e0140e8bf941399a17d34e2` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:10:24.200300 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abcc06c4-1147-4f49-890a-dee941692df9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=75daef2c3e0140e8bf941399a17d34e2)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:10:24.204870 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abcc06c4-1147-4f49-890a-dee941692df9 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:10:24.272461 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:10:24.511387 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-abcc06c4-1147-4f49-890a-dee941692df9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '75daef2c3e0140e8bf941399a17d34e2'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:10:24.535495 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-abcc06c4-1147-4f49-890a-dee941692df9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '75daef2c3e0140e8bf941399a17d34e2'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:10:24.537952 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-abcc06c4-1147-4f49-890a-dee941692df9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '75daef2c3e0140e8bf941399a17d34e2'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:10:24.628886 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-abcc06c4-1147-4f49-890a-dee941692df9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '75daef2c3e0140e8bf941399a17d34e2'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:10:24.652136 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-abcc06c4-1147-4f49-890a-dee941692df9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '75daef2c3e0140e8bf941399a17d34e2'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:10:24.674471 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1101/2177] 10.222.0.46 () {58 vars in 1224 bytes} [Mon Oct 10 12:10:24 2022] DELETE /identity/v3/users/75daef2c3e0140e8bf941399a17d34e2 => generated 0 bytes in 627 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:10:25.573078 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fcfdf6-0374-4511-8f62-59e1206e7e18 admin admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:10:25.575524 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-e5fcfdf6-0374-4511-8f62-59e1206e7e18 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:10:25.606940 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fcfdf6-0374-4511-8f62-59e1206e7e18 admin admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:10:25.742194 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:10:25.755632 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57027bd0-cfc9-4296-a26b-6bc426b8614f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:10:25.758711 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57027bd0-cfc9-4296-a26b-6bc426b8614f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:10:25.759469 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57027bd0-cfc9-4296-a26b-6bc426b8614f admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:10:25.759986 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57027bd0-cfc9-4296-a26b-6bc426b8614f admin admin] PATH_INFO: `/v3/projects/ad57ccf6538b449fb6a866f4322a027c` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:10:25.761597 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57027bd0-cfc9-4296-a26b-6bc426b8614f admin admin] RBAC: Authorizing `identity:delete_project(project_id=ad57ccf6538b449fb6a866f4322a027c)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:10:25.764248 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57027bd0-cfc9-4296-a26b-6bc426b8614f admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:10:25.959303 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-57027bd0-cfc9-4296-a26b-6bc426b8614f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ad57ccf6538b449fb6a866f4322a027c', 'request_id': 'req-57027bd0-cfc9-4296-a26b-6bc426b8614f'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:10:25.968512 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-57027bd0-cfc9-4296-a26b-6bc426b8614f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ad57ccf6538b449fb6a866f4322a027c', 'request_id': 'req-57027bd0-cfc9-4296-a26b-6bc426b8614f'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:10:25.987679 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-57027bd0-cfc9-4296-a26b-6bc426b8614f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ad57ccf6538b449fb6a866f4322a027c', 'request_id': 'req-57027bd0-cfc9-4296-a26b-6bc426b8614f'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:10:26.153691 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-57027bd0-cfc9-4296-a26b-6bc426b8614f admin admin] The token cache is being invalidate because project ad57ccf6538b449fb6a866f4322a027c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81676) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 10 12:10:26.154431 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-57027bd0-cfc9-4296-a26b-6bc426b8614f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:10:26.160116 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-57027bd0-cfc9-4296-a26b-6bc426b8614f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:10:26.160116 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1077/2178] 10.222.0.46 () {58 vars in 1233 bytes} [Mon Oct 10 12:10:25 2022] DELETE /identity/v3/projects/ad57ccf6538b449fb6a866f4322a027c => generated 0 bytes in 588 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:10:26.167804 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8586a48-588e-4d54-b8e7-125dc44ec561 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:10:26.168573 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8586a48-588e-4d54-b8e7-125dc44ec561 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:10:26.168573 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8586a48-588e-4d54-b8e7-125dc44ec561 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:10:26.231857 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-a8586a48-588e-4d54-b8e7-125dc44ec561 None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:10:26.235186 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-a8586a48-588e-4d54-b8e7-125dc44ec561 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:10:26.461876 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1102/2179] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:10:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 282 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:10:26.475179 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0bc484-b089-46a5-b39c-0f619fcd057c None None] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:10:26.475179 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0bc484-b089-46a5-b39c-0f619fcd057c None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:10:26.475179 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0bc484-b089-46a5-b39c-0f619fcd057c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:10:26.522431 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.auth.core [None req-8f0bc484-b089-46a5-b39c-0f619fcd057c None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:10:26.524011 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-8f0bc484-b089-46a5-b39c-0f619fcd057c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:10:26.534435 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1078/2180] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:10:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:10:26.542318 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8586a48-588e-4d54-b8e7-125dc44ec561 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:10:26.576700 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8586a48-588e-4d54-b8e7-125dc44ec561 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:10:26.618720 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3889ca-fbd1-4e34-804b-e9e3cb9c7555 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:10:26.619562 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3889ca-fbd1-4e34-804b-e9e3cb9c7555 admin admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:10:26.620562 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3889ca-fbd1-4e34-804b-e9e3cb9c7555 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:10:26.620770 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3889ca-fbd1-4e34-804b-e9e3cb9c7555 admin admin] PATH_INFO: `/v3/domains` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:10:26.623406 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3889ca-fbd1-4e34-804b-e9e3cb9c7555 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 12:10:26.623406 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3889ca-fbd1-4e34-804b-e9e3cb9c7555 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:10:26.629915 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3889ca-fbd1-4e34-804b-e9e3cb9c7555 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:10:26.660064 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1103/2181] 10.222.0.46 () {58 vars in 1153 bytes} [Mon Oct 10 12:10:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 12:10:26.678966 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0bc484-b089-46a5-b39c-0f619fcd057c None None] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:10:26.724270 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0bc484-b089-46a5-b39c-0f619fcd057c None None] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:10:26.744460 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb412da-d03a-4429-ab58-6e92ec05d21c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:10:26.746531 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb412da-d03a-4429-ab58-6e92ec05d21c admin admin] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:10:26.747116 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb412da-d03a-4429-ab58-6e92ec05d21c admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:10:26.747446 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb412da-d03a-4429-ab58-6e92ec05d21c admin admin] PATH_INFO: `/v3/projects` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:10:26.748297 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb412da-d03a-4429-ab58-6e92ec05d21c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:10:26.750495 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb412da-d03a-4429-ab58-6e92ec05d21c admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:10:26.918831 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1079/2182] 10.222.0.46 () {60 vars in 1153 bytes} [Mon Oct 10 12:10:26 2022] POST /identity/v3/projects => generated 387 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 12:10:26.921170 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3889ca-fbd1-4e34-804b-e9e3cb9c7555 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:10:26.948240 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3889ca-fbd1-4e34-804b-e9e3cb9c7555 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:10:27.065686 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec40ee3-cfe5-4f5c-8006-3de767c4cfb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:10:27.066979 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec40ee3-cfe5-4f5c-8006-3de767c4cfb6 admin admin] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:10:27.067680 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec40ee3-cfe5-4f5c-8006-3de767c4cfb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:10:27.068293 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec40ee3-cfe5-4f5c-8006-3de767c4cfb6 admin admin] PATH_INFO: `/v3/users` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:10:27.069540 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec40ee3-cfe5-4f5c-8006-3de767c4cfb6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:10:27.073570 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec40ee3-cfe5-4f5c-8006-3de767c4cfb6 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:10:27.268129 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1104/2183] 10.222.0.46 () {60 vars in 1144 bytes} [Mon Oct 10 12:10:26 2022] POST /identity/v3/users => generated 294 bytes in 348 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 12:10:27.272578 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb412da-d03a-4429-ab58-6e92ec05d21c admin admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:10:27.316791 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb412da-d03a-4429-ab58-6e92ec05d21c admin admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:10:27.327931 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bddd23e5-e650-42c1-8d2a-482d35898976 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:10:27.329942 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddd23e5-e650-42c1-8d2a-482d35898976 admin admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:10:27.330833 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddd23e5-e650-42c1-8d2a-482d35898976 admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:10:27.332123 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddd23e5-e650-42c1-8d2a-482d35898976 admin admin] PATH_INFO: `/v3/roles` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:10:27.333750 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bddd23e5-e650-42c1-8d2a-482d35898976 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:10:27.337543 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bddd23e5-e650-42c1-8d2a-482d35898976 admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:10:27.369064 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1080/2184] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 10 12:10:27 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 10 12:10:27.381240 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec40ee3-cfe5-4f5c-8006-3de767c4cfb6 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:10:27.402410 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec40ee3-cfe5-4f5c-8006-3de767c4cfb6 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:10:27.419723 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080095ca-d3be-446a-89eb-f628f2926587 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:10:27.420779 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080095ca-d3be-446a-89eb-f628f2926587 admin admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:10:27.421123 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080095ca-d3be-446a-89eb-f628f2926587 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:10:27.421524 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080095ca-d3be-446a-89eb-f628f2926587 admin admin] PATH_INFO: `/v3/roles` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:10:27.422538 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080095ca-d3be-446a-89eb-f628f2926587 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:10:27.425886 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080095ca-d3be-446a-89eb-f628f2926587 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:10:27.445997 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1105/2185] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 10 12:10:27 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 12:10:27.451222 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bddd23e5-e650-42c1-8d2a-482d35898976 admin admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:10:27.475043 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bddd23e5-e650-42c1-8d2a-482d35898976 admin admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:10:27.494885 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02fd8b8-b453-47fc-9ea5-928fa8b2edfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:10:27.496220 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02fd8b8-b453-47fc-9ea5-928fa8b2edfe admin admin] REQUEST_METHOD: `PUT` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:10:27.496692 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02fd8b8-b453-47fc-9ea5-928fa8b2edfe admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:10:27.497161 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02fd8b8-b453-47fc-9ea5-928fa8b2edfe admin admin] PATH_INFO: `/v3/projects/8742e57736944b3e853daaf76b10393b/users/59104f3e1f82448d851e2d5fb698f41f/roles/b2cb2a6e818e47338ce071e6fedfc673` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:10:27.525243 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02fd8b8-b453-47fc-9ea5-928fa8b2edfe admin admin] RBAC: Authorizing `identity:create_grant(project_id=8742e57736944b3e853daaf76b10393b, user_id=59104f3e1f82448d851e2d5fb698f41f, role_id=b2cb2a6e818e47338ce071e6fedfc673)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:10:27.531387 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02fd8b8-b453-47fc-9ea5-928fa8b2edfe admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:10:27.634812 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1081/2186] 10.222.0.46 () {60 vars in 1483 bytes} [Mon Oct 10 12:10:27 2022] PUT /identity/v3/projects/8742e57736944b3e853daaf76b10393b/users/59104f3e1f82448d851e2d5fb698f41f/roles/b2cb2a6e818e47338ce071e6fedfc673 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 10 12:10:27.645205 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66ccb90-254a-4cf5-aa7d-84eec5b98d92 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:10:27.645682 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66ccb90-254a-4cf5-aa7d-84eec5b98d92 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:10:27.646179 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66ccb90-254a-4cf5-aa7d-84eec5b98d92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:10:27.673564 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b31b177-9fe9-4afd-b6d2-0fd713cc26b0 None None] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:10:27.674042 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b31b177-9fe9-4afd-b6d2-0fd713cc26b0 None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:10:27.674492 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b31b177-9fe9-4afd-b6d2-0fd713cc26b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:10:27.681553 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:10:27.713966 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-c66ccb90-254a-4cf5-aa7d-84eec5b98d92 None None] MFA Rules not processed for user `59104f3e1f82448d851e2d5fb698f41f`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:10:27.716381 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-c66ccb90-254a-4cf5-aa7d-84eec5b98d92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:10:27.731865 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.auth.core [None req-8b31b177-9fe9-4afd-b6d2-0fd713cc26b0 None None] MFA Rules not processed for user `09668960625845f0ad79bb91a6de7766`. Rule list: `[]` (Enabled: `True`). {{(pid=81676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:10:27.735102 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-8b31b177-9fe9-4afd-b6d2-0fd713cc26b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:10:27.830851 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1106/2187] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:10:27 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:10:27.836235 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66ccb90-254a-4cf5-aa7d-84eec5b98d92 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:10:27.887395 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1082/2188] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:10:27 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:10:27.898164 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b31b177-9fe9-4afd-b6d2-0fd713cc26b0 None None] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:10:27.901995 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-8b31b177-9fe9-4afd-b6d2-0fd713cc26b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:10:27.929229 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:10:27.952099 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66ccb90-254a-4cf5-aa7d-84eec5b98d92 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:10:27.959296 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbcd722b-19db-46e9-9d91-1fae7b9408e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:10:27.960325 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcd722b-19db-46e9-9d91-1fae7b9408e6 admin admin] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:10:27.960774 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcd722b-19db-46e9-9d91-1fae7b9408e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:10:27.961245 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcd722b-19db-46e9-9d91-1fae7b9408e6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:10:27.962226 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbcd722b-19db-46e9-9d91-1fae7b9408e6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:10:27.965200 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbcd722b-19db-46e9-9d91-1fae7b9408e6 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:10:28.019806 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b31b177-9fe9-4afd-b6d2-0fd713cc26b0 None None] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:10:28.026246 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70833827-b7ff-4582-856b-b22bea6aeab4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f130bbb69e546ce99b4e14de81d7e54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad3772f7b6a47c28b7ba37a9316a3c1', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:10:28.028139 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70833827-b7ff-4582-856b-b22bea6aeab4 service nova] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:10:28.028139 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70833827-b7ff-4582-856b-b22bea6aeab4 service nova] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:10:28.028139 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70833827-b7ff-4582-856b-b22bea6aeab4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:10:28.031657 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-70833827-b7ff-4582-856b-b22bea6aeab4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:10:28.119346 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1107/2189] 10.222.0.46 () {60 vars in 1153 bytes} [Mon Oct 10 12:10:27 2022] POST /identity/v3/projects => generated 385 bytes in 279 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 12:10:28.120919 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbcd722b-19db-46e9-9d91-1fae7b9408e6 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:10:28.134324 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70833827-b7ff-4582-856b-b22bea6aeab4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:10:28.138684 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70833827-b7ff-4582-856b-b22bea6aeab4 service nova] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:10:28.161904 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1083/2190] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 10 12:10:27 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 265 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 10 12:10:28.247672 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbcd722b-19db-46e9-9d91-1fae7b9408e6 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:10:28.259704 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cea35a4-355a-4663-9021-9bb1e888d507 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:10:28.261171 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cea35a4-355a-4663-9021-9bb1e888d507 admin admin] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:10:28.261771 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cea35a4-355a-4663-9021-9bb1e888d507 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:10:28.262265 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cea35a4-355a-4663-9021-9bb1e888d507 admin admin] PATH_INFO: `/v3/users` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:10:28.263601 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cea35a4-355a-4663-9021-9bb1e888d507 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:10:28.268489 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cea35a4-355a-4663-9021-9bb1e888d507 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:10:28.443848 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1108/2191] 10.222.0.46 () {60 vars in 1144 bytes} [Mon Oct 10 12:10:28 2022] POST /identity/v3/users => generated 299 bytes in 324 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 10 12:10:28.449164 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70833827-b7ff-4582-856b-b22bea6aeab4 service nova] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:10:28.471828 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70833827-b7ff-4582-856b-b22bea6aeab4 service nova] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:10:28.483453 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf648cb0-ee89-4daa-a0ce-59bb5db6c3d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:10:28.490811 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf648cb0-ee89-4daa-a0ce-59bb5db6c3d4 admin admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:10:28.491325 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf648cb0-ee89-4daa-a0ce-59bb5db6c3d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:10:28.491805 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf648cb0-ee89-4daa-a0ce-59bb5db6c3d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:10:28.493094 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf648cb0-ee89-4daa-a0ce-59bb5db6c3d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:10:28.497606 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf648cb0-ee89-4daa-a0ce-59bb5db6c3d4 admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:10:28.525189 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1084/2192] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 10 12:10:28 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 10 12:10:28.531563 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cea35a4-355a-4663-9021-9bb1e888d507 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:10:28.542962 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cea35a4-355a-4663-9021-9bb1e888d507 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:10:28.549253 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81636d2c-f341-4239-a191-de23cb7d03a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:10:28.550203 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81636d2c-f341-4239-a191-de23cb7d03a2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:10:28.551196 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81636d2c-f341-4239-a191-de23cb7d03a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:10:28.551667 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81636d2c-f341-4239-a191-de23cb7d03a2 admin admin] PATH_INFO: `/v3/projects/5f31172e3a174a1992bce1e79f5be92d/users/172301ea6bb54be4ada47b7af6147dce/roles/0d822a0674b6495fac623877a3b38025` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:10:28.556405 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:10:28.575574 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81636d2c-f341-4239-a191-de23cb7d03a2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5f31172e3a174a1992bce1e79f5be92d, user_id=172301ea6bb54be4ada47b7af6147dce, role_id=0d822a0674b6495fac623877a3b38025)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:10:28.579393 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81636d2c-f341-4239-a191-de23cb7d03a2 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:10:28.677547 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1109/2193] 10.222.0.46 () {60 vars in 1483 bytes} [Mon Oct 10 12:10:28 2022] PUT /identity/v3/projects/5f31172e3a174a1992bce1e79f5be92d/users/172301ea6bb54be4ada47b7af6147dce/roles/0d822a0674b6495fac623877a3b38025 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:10:28.683045 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf648cb0-ee89-4daa-a0ce-59bb5db6c3d4 admin admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:10:28.696272 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf648cb0-ee89-4daa-a0ce-59bb5db6c3d4 admin admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:10:28.819066 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7a0bc8-bb96-4092-9cd2-96206e08e937 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:10:28.821208 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7a0bc8-bb96-4092-9cd2-96206e08e937 admin admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:10:28.821729 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7a0bc8-bb96-4092-9cd2-96206e08e937 admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:10:28.822189 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7a0bc8-bb96-4092-9cd2-96206e08e937 admin admin] PATH_INFO: `/v3/roles` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:10:28.823318 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7a0bc8-bb96-4092-9cd2-96206e08e937 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:10:28.828132 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7a0bc8-bb96-4092-9cd2-96206e08e937 admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:10:28.847295 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1085/2194] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 10 12:10:28 2022] GET /identity/v3/roles => generated 1420 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 10 12:10:28.849083 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81636d2c-f341-4239-a191-de23cb7d03a2 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:10:28.861902 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81636d2c-f341-4239-a191-de23cb7d03a2 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:10:28.867462 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c9a4f8-3238-4268-8bd0-0f2bb68aeacc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:10:28.868377 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c9a4f8-3238-4268-8bd0-0f2bb68aeacc admin admin] REQUEST_METHOD: `PUT` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:10:28.868825 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c9a4f8-3238-4268-8bd0-0f2bb68aeacc admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:10:28.869202 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c9a4f8-3238-4268-8bd0-0f2bb68aeacc admin admin] PATH_INFO: `/v3/projects/5f31172e3a174a1992bce1e79f5be92d/users/172301ea6bb54be4ada47b7af6147dce/roles/0d822a0674b6495fac623877a3b38025` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:10:28.872461 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c9a4f8-3238-4268-8bd0-0f2bb68aeacc admin admin] RBAC: Authorizing `identity:create_grant(project_id=5f31172e3a174a1992bce1e79f5be92d, user_id=172301ea6bb54be4ada47b7af6147dce, role_id=0d822a0674b6495fac623877a3b38025)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:10:28.877572 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c9a4f8-3238-4268-8bd0-0f2bb68aeacc admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:10:28.892676 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1110/2195] 10.222.0.46 () {60 vars in 1483 bytes} [Mon Oct 10 12:10:28 2022] PUT /identity/v3/projects/5f31172e3a174a1992bce1e79f5be92d/users/172301ea6bb54be4ada47b7af6147dce/roles/0d822a0674b6495fac623877a3b38025 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 10 12:10:28.911472 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e29f305-f04a-42ea-99c9-3895871e072d None None] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:10:28.911793 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e29f305-f04a-42ea-99c9-3895871e072d None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:10:28.912179 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e29f305-f04a-42ea-99c9-3895871e072d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:10:28.971395 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.auth.core [None req-2e29f305-f04a-42ea-99c9-3895871e072d None None] MFA Rules not processed for user `172301ea6bb54be4ada47b7af6147dce`. Rule list: `[]` (Enabled: `True`). {{(pid=81676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:10:28.974356 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-2e29f305-f04a-42ea-99c9-3895871e072d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:10:29.006341 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:10:29.118058 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1086/2196] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:10:28 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:10:29.133364 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c9a4f8-3238-4268-8bd0-0f2bb68aeacc admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:10:29.135351 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-53c9a4f8-3238-4268-8bd0-0f2bb68aeacc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:10:29.241256 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c9a4f8-3238-4268-8bd0-0f2bb68aeacc admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:10:29.254459 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0daf78a-eef7-495d-9644-0c624acfc880 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3ae093470484b93b23dcfae1657df9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad3772f7b6a47c28b7ba37a9316a3c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:10:29.254808 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0daf78a-eef7-495d-9644-0c624acfc880 service cinder] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:10:29.255283 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0daf78a-eef7-495d-9644-0c624acfc880 service cinder] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:10:29.255849 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0daf78a-eef7-495d-9644-0c624acfc880 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:10:29.274987 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0daf78a-eef7-495d-9644-0c624acfc880 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:10:29.277832 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0daf78a-eef7-495d-9644-0c624acfc880 service cinder] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:10:29.301183 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1111/2197] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 10 12:10:29 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:10:29.333982 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e29f305-f04a-42ea-99c9-3895871e072d None None] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:10:29.422421 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e29f305-f04a-42ea-99c9-3895871e072d None None] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:10:29.427793 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96530b9d-8857-46a5-997e-a04f6f5310d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:10:29.428450 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96530b9d-8857-46a5-997e-a04f6f5310d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:10:29.428734 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96530b9d-8857-46a5-997e-a04f6f5310d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:10:29.429014 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96530b9d-8857-46a5-997e-a04f6f5310d3 admin admin] PATH_INFO: `/v3/users/59104f3e1f82448d851e2d5fb698f41f` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:10:29.430212 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96530b9d-8857-46a5-997e-a04f6f5310d3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=59104f3e1f82448d851e2d5fb698f41f)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:10:29.432390 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96530b9d-8857-46a5-997e-a04f6f5310d3 admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:10:29.714895 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-96530b9d-8857-46a5-997e-a04f6f5310d3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '59104f3e1f82448d851e2d5fb698f41f'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:10:29.731242 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-96530b9d-8857-46a5-997e-a04f6f5310d3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '59104f3e1f82448d851e2d5fb698f41f'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:10:29.732880 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-96530b9d-8857-46a5-997e-a04f6f5310d3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '59104f3e1f82448d851e2d5fb698f41f'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:10:29.829302 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-96530b9d-8857-46a5-997e-a04f6f5310d3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '59104f3e1f82448d851e2d5fb698f41f'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:10:29.851682 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-96530b9d-8857-46a5-997e-a04f6f5310d3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '59104f3e1f82448d851e2d5fb698f41f'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:10:29.859639 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1087/2198] 10.222.0.46 () {58 vars in 1224 bytes} [Mon Oct 10 12:10:29 2022] DELETE /identity/v3/users/59104f3e1f82448d851e2d5fb698f41f => generated 0 bytes in 526 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:10:29.869387 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfeaa324-6b68-4441-b1ab-ac1515fa56a8 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:10:29.869709 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfeaa324-6b68-4441-b1ab-ac1515fa56a8 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:10:29.870059 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfeaa324-6b68-4441-b1ab-ac1515fa56a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:10:29.891957 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-bfeaa324-6b68-4441-b1ab-ac1515fa56a8 None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:10:29.893993 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-bfeaa324-6b68-4441-b1ab-ac1515fa56a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:10:29.991770 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1112/2199] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:10:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:10:30.007865 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96530b9d-8857-46a5-997e-a04f6f5310d3 admin admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:10:30.010703 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-96530b9d-8857-46a5-997e-a04f6f5310d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:10:30.206622 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:10:30.236133 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96530b9d-8857-46a5-997e-a04f6f5310d3 admin admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:10:30.245821 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b475a78f-0351-4050-8482-f9162eb0deb3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58b1c7c85302421cba823ace4201861f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad3772f7b6a47c28b7ba37a9316a3c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:10:30.246710 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b475a78f-0351-4050-8482-f9162eb0deb3 service neutron] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:10:30.247141 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b475a78f-0351-4050-8482-f9162eb0deb3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:10:30.247561 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b475a78f-0351-4050-8482-f9162eb0deb3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:10:30.270149 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b475a78f-0351-4050-8482-f9162eb0deb3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:10:30.272542 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b475a78f-0351-4050-8482-f9162eb0deb3 service neutron] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:10:30.286502 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1088/2200] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 10 12:10:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 280 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:10:31.003928 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfeaa324-6b68-4441-b1ab-ac1515fa56a8 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:10:31.007301 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-bfeaa324-6b68-4441-b1ab-ac1515fa56a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:10:31.043812 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfeaa324-6b68-4441-b1ab-ac1515fa56a8 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:10:31.057902 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20f08a3-4150-4a46-b152-5721fb5814a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:10:31.058988 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20f08a3-4150-4a46-b152-5721fb5814a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:10:31.059467 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20f08a3-4150-4a46-b152-5721fb5814a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:10:31.059953 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20f08a3-4150-4a46-b152-5721fb5814a0 admin admin] PATH_INFO: `/v3/projects/8742e57736944b3e853daaf76b10393b` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:10:31.062149 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20f08a3-4150-4a46-b152-5721fb5814a0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8742e57736944b3e853daaf76b10393b)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:10:31.066718 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20f08a3-4150-4a46-b152-5721fb5814a0 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:10:31.223977 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-e20f08a3-4150-4a46-b152-5721fb5814a0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8742e57736944b3e853daaf76b10393b', 'request_id': 'req-e20f08a3-4150-4a46-b152-5721fb5814a0'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:10:31.236625 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-e20f08a3-4150-4a46-b152-5721fb5814a0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8742e57736944b3e853daaf76b10393b', 'request_id': 'req-e20f08a3-4150-4a46-b152-5721fb5814a0'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:10:31.249112 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-e20f08a3-4150-4a46-b152-5721fb5814a0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8742e57736944b3e853daaf76b10393b', 'request_id': 'req-e20f08a3-4150-4a46-b152-5721fb5814a0'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:10:31.356038 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:10:31.365265 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-e20f08a3-4150-4a46-b152-5721fb5814a0 admin admin] The token cache is being invalidate because project 8742e57736944b3e853daaf76b10393b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81677) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 10 12:10:31.365774 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-e20f08a3-4150-4a46-b152-5721fb5814a0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:10:31.366605 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-e20f08a3-4150-4a46-b152-5721fb5814a0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:10:31.368648 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1113/2201] 10.222.0.46 () {58 vars in 1233 bytes} [Mon Oct 10 12:10:31 2022] DELETE /identity/v3/projects/8742e57736944b3e853daaf76b10393b => generated 0 bytes in 365 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:10:31.373606 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b475a78f-0351-4050-8482-f9162eb0deb3 service neutron] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:10:31.387369 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b475a78f-0351-4050-8482-f9162eb0deb3 service neutron] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:10:31.525712 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26539c84-ee9c-4521-bbc8-99e3bc4ad889 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:10:31.526510 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26539c84-ee9c-4521-bbc8-99e3bc4ad889 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:10:31.526929 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26539c84-ee9c-4521-bbc8-99e3bc4ad889 admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:10:31.527303 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26539c84-ee9c-4521-bbc8-99e3bc4ad889 admin admin] PATH_INFO: `/v3/users/172301ea6bb54be4ada47b7af6147dce` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:10:31.528682 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26539c84-ee9c-4521-bbc8-99e3bc4ad889 admin admin] RBAC: Authorizing `identity:delete_user(user_id=172301ea6bb54be4ada47b7af6147dce)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:10:31.530937 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26539c84-ee9c-4521-bbc8-99e3bc4ad889 admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:10:31.825256 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-26539c84-ee9c-4521-bbc8-99e3bc4ad889 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '172301ea6bb54be4ada47b7af6147dce'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:10:31.843896 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-26539c84-ee9c-4521-bbc8-99e3bc4ad889 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '172301ea6bb54be4ada47b7af6147dce'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:10:31.847263 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-26539c84-ee9c-4521-bbc8-99e3bc4ad889 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '172301ea6bb54be4ada47b7af6147dce'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:10:32.012329 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-26539c84-ee9c-4521-bbc8-99e3bc4ad889 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '172301ea6bb54be4ada47b7af6147dce'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:10:32.031334 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-26539c84-ee9c-4521-bbc8-99e3bc4ad889 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '172301ea6bb54be4ada47b7af6147dce'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:10:32.039400 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1089/2202] 10.222.0.46 () {58 vars in 1224 bytes} [Mon Oct 10 12:10:31 2022] DELETE /identity/v3/users/172301ea6bb54be4ada47b7af6147dce => generated 0 bytes in 666 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:10:32.799630 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20f08a3-4150-4a46-b152-5721fb5814a0 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:10:32.803117 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-e20f08a3-4150-4a46-b152-5721fb5814a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:10:32.855059 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20f08a3-4150-4a46-b152-5721fb5814a0 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:10:32.960061 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236c4986-4cf1-4b84-aa4e-26cf225ddde8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:10:32.960732 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236c4986-4cf1-4b84-aa4e-26cf225ddde8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:10:32.961933 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236c4986-4cf1-4b84-aa4e-26cf225ddde8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:10:32.961933 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236c4986-4cf1-4b84-aa4e-26cf225ddde8 admin admin] PATH_INFO: `/v3/projects/5f31172e3a174a1992bce1e79f5be92d` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:10:32.962494 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236c4986-4cf1-4b84-aa4e-26cf225ddde8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5f31172e3a174a1992bce1e79f5be92d)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:10:32.964558 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236c4986-4cf1-4b84-aa4e-26cf225ddde8 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:10:33.116035 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-236c4986-4cf1-4b84-aa4e-26cf225ddde8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5f31172e3a174a1992bce1e79f5be92d', 'request_id': 'req-236c4986-4cf1-4b84-aa4e-26cf225ddde8'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:10:33.127005 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-236c4986-4cf1-4b84-aa4e-26cf225ddde8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5f31172e3a174a1992bce1e79f5be92d', 'request_id': 'req-236c4986-4cf1-4b84-aa4e-26cf225ddde8'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:10:33.137067 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-236c4986-4cf1-4b84-aa4e-26cf225ddde8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5f31172e3a174a1992bce1e79f5be92d', 'request_id': 'req-236c4986-4cf1-4b84-aa4e-26cf225ddde8'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:10:33.229547 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-236c4986-4cf1-4b84-aa4e-26cf225ddde8 admin admin] The token cache is being invalidate because project 5f31172e3a174a1992bce1e79f5be92d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81677) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 10 12:10:33.229929 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-236c4986-4cf1-4b84-aa4e-26cf225ddde8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:10:33.230818 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-236c4986-4cf1-4b84-aa4e-26cf225ddde8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:10:33.232690 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1114/2203] 10.222.0.46 () {58 vars in 1233 bytes} [Mon Oct 10 12:10:32 2022] DELETE /identity/v3/projects/5f31172e3a174a1992bce1e79f5be92d => generated 0 bytes in 435 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:10:33.241153 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5aaf179-a3a5-4e81-9b74-95ad7d31f731 None None] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:10:33.242650 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5aaf179-a3a5-4e81-9b74-95ad7d31f731 None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:10:33.243219 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5aaf179-a3a5-4e81-9b74-95ad7d31f731 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:10:33.269668 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.auth.core [None req-b5aaf179-a3a5-4e81-9b74-95ad7d31f731 None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:10:33.270918 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-b5aaf179-a3a5-4e81-9b74-95ad7d31f731 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:10:33.335766 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:10:33.346891 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1090/2204] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:10:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:10:33.357699 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3fadcf-cf24-48ac-9bcc-b486c1c2d2ad None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:10:33.358019 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3fadcf-cf24-48ac-9bcc-b486c1c2d2ad None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:10:33.358249 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3fadcf-cf24-48ac-9bcc-b486c1c2d2ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:10:33.381043 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-7f3fadcf-cf24-48ac-9bcc-b486c1c2d2ad None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:10:33.382277 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-7f3fadcf-cf24-48ac-9bcc-b486c1c2d2ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:10:33.391431 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1115/2205] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:10:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:10:33.396865 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5aaf179-a3a5-4e81-9b74-95ad7d31f731 None None] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:10:33.411607 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5aaf179-a3a5-4e81-9b74-95ad7d31f731 None None] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:10:33.416837 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da1cd0e-c6e7-43cd-bf53-f03bad4dd631 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:10:33.418898 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da1cd0e-c6e7-43cd-bf53-f03bad4dd631 admin admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:10:33.418898 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da1cd0e-c6e7-43cd-bf53-f03bad4dd631 admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:10:33.418898 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da1cd0e-c6e7-43cd-bf53-f03bad4dd631 admin admin] PATH_INFO: `/v3/domains` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:10:33.418898 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da1cd0e-c6e7-43cd-bf53-f03bad4dd631 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 12:10:33.419323 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da1cd0e-c6e7-43cd-bf53-f03bad4dd631 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:10:33.420662 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da1cd0e-c6e7-43cd-bf53-f03bad4dd631 admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:10:33.434168 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1091/2206] 10.222.0.46 () {58 vars in 1153 bytes} [Mon Oct 10 12:10:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 12:10:33.439248 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3fadcf-cf24-48ac-9bcc-b486c1c2d2ad None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:10:33.452923 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3fadcf-cf24-48ac-9bcc-b486c1c2d2ad None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:10:33.471246 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05107825-821a-49be-a910-c98ac1251ce6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:10:33.471246 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05107825-821a-49be-a910-c98ac1251ce6 admin admin] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:10:33.471246 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05107825-821a-49be-a910-c98ac1251ce6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:10:33.471246 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05107825-821a-49be-a910-c98ac1251ce6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:10:33.471610 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05107825-821a-49be-a910-c98ac1251ce6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:10:33.471610 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05107825-821a-49be-a910-c98ac1251ce6 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:10:33.612620 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1116/2207] 10.222.0.46 () {60 vars in 1153 bytes} [Mon Oct 10 12:10:33 2022] POST /identity/v3/projects => generated 369 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 12:10:33.617587 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da1cd0e-c6e7-43cd-bf53-f03bad4dd631 admin admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:10:33.629532 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da1cd0e-c6e7-43cd-bf53-f03bad4dd631 admin admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:10:33.714460 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e20e02-4581-466e-8ab5-75d95d1975b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:10:33.715289 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e20e02-4581-466e-8ab5-75d95d1975b0 admin admin] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:10:33.715617 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e20e02-4581-466e-8ab5-75d95d1975b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:10:33.715953 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e20e02-4581-466e-8ab5-75d95d1975b0 admin admin] PATH_INFO: `/v3/users` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:10:33.716680 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e20e02-4581-466e-8ab5-75d95d1975b0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:10:33.719271 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e20e02-4581-466e-8ab5-75d95d1975b0 admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:10:33.867783 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1092/2208] 10.222.0.46 () {60 vars in 1144 bytes} [Mon Oct 10 12:10:33 2022] POST /identity/v3/users => generated 285 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 12:10:33.872412 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05107825-821a-49be-a910-c98ac1251ce6 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:10:33.884570 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05107825-821a-49be-a910-c98ac1251ce6 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:10:33.889600 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5752460-8bd9-4623-bb13-58a25a9a59e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:10:33.890550 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5752460-8bd9-4623-bb13-58a25a9a59e1 admin admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:10:33.890843 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5752460-8bd9-4623-bb13-58a25a9a59e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:10:33.891128 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5752460-8bd9-4623-bb13-58a25a9a59e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:10:33.891832 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5752460-8bd9-4623-bb13-58a25a9a59e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:10:33.894047 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5752460-8bd9-4623-bb13-58a25a9a59e1 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:10:33.909362 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1117/2209] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 10 12:10:33 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 10 12:10:33.915256 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e20e02-4581-466e-8ab5-75d95d1975b0 admin admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:10:33.926801 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e20e02-4581-466e-8ab5-75d95d1975b0 admin admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:10:33.932098 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecdb926-1e43-4d71-8a05-e641003920ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:10:33.932744 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecdb926-1e43-4d71-8a05-e641003920ea admin admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:10:33.933028 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecdb926-1e43-4d71-8a05-e641003920ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:10:33.933313 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecdb926-1e43-4d71-8a05-e641003920ea admin admin] PATH_INFO: `/v3/roles` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:10:33.933996 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fecdb926-1e43-4d71-8a05-e641003920ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:10:33.935738 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fecdb926-1e43-4d71-8a05-e641003920ea admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:10:33.949739 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1093/2210] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 10 12:10:33 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 10 12:10:33.956256 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5752460-8bd9-4623-bb13-58a25a9a59e1 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:10:33.972024 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5752460-8bd9-4623-bb13-58a25a9a59e1 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:10:33.977408 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6333962f-d02d-42f6-ae6a-b2a64d8c17e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:10:33.978114 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6333962f-d02d-42f6-ae6a-b2a64d8c17e9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:10:33.978403 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6333962f-d02d-42f6-ae6a-b2a64d8c17e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:10:33.978713 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6333962f-d02d-42f6-ae6a-b2a64d8c17e9 admin admin] PATH_INFO: `/v3/projects/11848252824f47d295158d07ce2305cf/users/821897e0ad4a47b2accd15144b5ef9eb/roles/b2cb2a6e818e47338ce071e6fedfc673` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:10:33.997444 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6333962f-d02d-42f6-ae6a-b2a64d8c17e9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=11848252824f47d295158d07ce2305cf, user_id=821897e0ad4a47b2accd15144b5ef9eb, role_id=b2cb2a6e818e47338ce071e6fedfc673)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:10:34.000444 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6333962f-d02d-42f6-ae6a-b2a64d8c17e9 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:10:34.095828 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1118/2211] 10.222.0.46 () {60 vars in 1483 bytes} [Mon Oct 10 12:10:33 2022] PUT /identity/v3/projects/11848252824f47d295158d07ce2305cf/users/821897e0ad4a47b2accd15144b5ef9eb/roles/b2cb2a6e818e47338ce071e6fedfc673 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:10:34.112697 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e50d517-0dab-4835-8730-1b5b33492005 None None] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:10:34.113167 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e50d517-0dab-4835-8730-1b5b33492005 None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:10:34.113494 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e50d517-0dab-4835-8730-1b5b33492005 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:10:34.159416 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.auth.core [None req-5e50d517-0dab-4835-8730-1b5b33492005 None None] MFA Rules not processed for user `821897e0ad4a47b2accd15144b5ef9eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:10:34.161618 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-5e50d517-0dab-4835-8730-1b5b33492005 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:10:34.309301 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1094/2212] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:10:34 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 10 12:10:34.331498 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6333962f-d02d-42f6-ae6a-b2a64d8c17e9 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:10:34.334444 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-6333962f-d02d-42f6-ae6a-b2a64d8c17e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:10:34.405338 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:10:34.455008 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6333962f-d02d-42f6-ae6a-b2a64d8c17e9 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:10:34.474620 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7443be46-2891-4bfd-a60b-2d94c19b6ae2 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '124f4aa8ab244ca2aadf4f2614df51e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad3772f7b6a47c28b7ba37a9316a3c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:10:34.476029 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7443be46-2891-4bfd-a60b-2d94c19b6ae2 service glance] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:10:34.479226 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7443be46-2891-4bfd-a60b-2d94c19b6ae2 service glance] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:10:34.480053 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7443be46-2891-4bfd-a60b-2d94c19b6ae2 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:10:34.504637 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7443be46-2891-4bfd-a60b-2d94c19b6ae2 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:10:34.510252 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7443be46-2891-4bfd-a60b-2d94c19b6ae2 service glance] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:10:34.523147 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1119/2213] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 10 12:10:34 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:10:37.883283 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b013dc-1c17-4451-914c-c2f3bdb5324c None None] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:10:37.884226 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b013dc-1c17-4451-914c-c2f3bdb5324c None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:10:37.884558 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b013dc-1c17-4451-914c-c2f3bdb5324c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:10:37.917036 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.auth.core [None req-26b013dc-1c17-4451-914c-c2f3bdb5324c None None] MFA Rules not processed for user `d58683f7689a4356badd8f4bb30850a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:10:37.927957 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-26b013dc-1c17-4451-914c-c2f3bdb5324c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:10:38.008368 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:10:38.017706 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1095/2214] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:10:37 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:10:44.694148 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=15 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:10:48.089914 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=1 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:10:53.092947 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7443be46-2891-4bfd-a60b-2d94c19b6ae2 service glance] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:10:53.094588 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-7443be46-2891-4bfd-a60b-2d94c19b6ae2 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:10:53.223708 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7443be46-2891-4bfd-a60b-2d94c19b6ae2 service glance] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:10:53.232693 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b2c286-9a4c-4718-b796-08afa0472203 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e8f9bb1776d47169ef7febb57c954cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad3772f7b6a47c28b7ba37a9316a3c1', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:10:53.233421 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b2c286-9a4c-4718-b796-08afa0472203 service placement] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:10:53.233751 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b2c286-9a4c-4718-b796-08afa0472203 service placement] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:10:53.234096 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b2c286-9a4c-4718-b796-08afa0472203 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:10:53.236110 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-a2b2c286-9a4c-4718-b796-08afa0472203 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:10:53.248980 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b2c286-9a4c-4718-b796-08afa0472203 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:10:53.250626 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b2c286-9a4c-4718-b796-08afa0472203 service placement] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:10:53.264546 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1120/2215] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 10 12:10:53 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:11:03.257392 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=32 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:11:39.393124 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2598efe6-9578-4471-adac-bdef2957134c None None] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:11:39.393710 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2598efe6-9578-4471-adac-bdef2957134c None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:11:39.393710 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2598efe6-9578-4471-adac-bdef2957134c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:11:39.415832 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.auth.core [None req-2598efe6-9578-4471-adac-bdef2957134c None None] MFA Rules not processed for user `21c537b0fefe4fb1b51483ac0e3cf2d1`. Rule list: `[]` (Enabled: `True`). {{(pid=81676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:11:39.417166 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-2598efe6-9578-4471-adac-bdef2957134c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:11:39.477333 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1096/2216] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:11:39 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:11:49.469724 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=25 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:11:58.397018 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b2c286-9a4c-4718-b796-08afa0472203 service placement] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:11:58.439870 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc2be88-d9c6-4bb5-aa55-681e9568fdf8 None None] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:11:58.440352 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc2be88-d9c6-4bb5-aa55-681e9568fdf8 None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:11:58.440601 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc2be88-d9c6-4bb5-aa55-681e9568fdf8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:11:58.499091 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.auth.core [None req-7dc2be88-d9c6-4bb5-aa55-681e9568fdf8 None None] MFA Rules not processed for user `fef422264d0c4a43a0aa1b9eff2251bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:11:58.500327 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-7dc2be88-d9c6-4bb5-aa55-681e9568fdf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:11:58.518710 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b2c286-9a4c-4718-b796-08afa0472203 service placement] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:11:58.523500 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253b38b0-5281-4444-b78c-7714abacfae8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:11:58.524148 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253b38b0-5281-4444-b78c-7714abacfae8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:11:58.524426 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253b38b0-5281-4444-b78c-7714abacfae8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:11:58.524695 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253b38b0-5281-4444-b78c-7714abacfae8 admin admin] PATH_INFO: `/v3/users/821897e0ad4a47b2accd15144b5ef9eb` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:11:58.525841 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253b38b0-5281-4444-b78c-7714abacfae8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=821897e0ad4a47b2accd15144b5ef9eb)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:11:58.528043 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253b38b0-5281-4444-b78c-7714abacfae8 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:11:58.584539 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1097/2217] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:11:58 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:11:58.796234 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-253b38b0-5281-4444-b78c-7714abacfae8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '821897e0ad4a47b2accd15144b5ef9eb'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:11:58.818567 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-253b38b0-5281-4444-b78c-7714abacfae8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '821897e0ad4a47b2accd15144b5ef9eb'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:11:58.819757 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-253b38b0-5281-4444-b78c-7714abacfae8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '821897e0ad4a47b2accd15144b5ef9eb'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:11:58.919500 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-253b38b0-5281-4444-b78c-7714abacfae8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '821897e0ad4a47b2accd15144b5ef9eb'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:11:58.940104 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-253b38b0-5281-4444-b78c-7714abacfae8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '821897e0ad4a47b2accd15144b5ef9eb'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:11:58.952251 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1121/2218] 10.222.0.46 () {58 vars in 1224 bytes} [Mon Oct 10 12:11:58 2022] DELETE /identity/v3/users/821897e0ad4a47b2accd15144b5ef9eb => generated 0 bytes in 556 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:11:58.960061 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3f0832-2cc0-46fe-8396-9b0614a19294 None None] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:11:58.960836 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3f0832-2cc0-46fe-8396-9b0614a19294 None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:11:58.961573 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3f0832-2cc0-46fe-8396-9b0614a19294 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:11:59.009701 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.auth.core [None req-1a3f0832-2cc0-46fe-8396-9b0614a19294 None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:11:59.012873 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-1a3f0832-2cc0-46fe-8396-9b0614a19294 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:11:59.135984 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1098/2219] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:11:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:11:59.163887 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253b38b0-5281-4444-b78c-7714abacfae8 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:11:59.168353 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-253b38b0-5281-4444-b78c-7714abacfae8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:11:59.258617 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253b38b0-5281-4444-b78c-7714abacfae8 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:11:59.266022 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8305b0-a492-4ed3-8383-482a67927aee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58b1c7c85302421cba823ace4201861f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad3772f7b6a47c28b7ba37a9316a3c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:11:59.271118 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8305b0-a492-4ed3-8383-482a67927aee service neutron] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:11:59.271118 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8305b0-a492-4ed3-8383-482a67927aee service neutron] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:11:59.271118 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8305b0-a492-4ed3-8383-482a67927aee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:11:59.290533 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8305b0-a492-4ed3-8383-482a67927aee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:11:59.295358 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8305b0-a492-4ed3-8383-482a67927aee service neutron] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:11:59.322094 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1122/2220] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 10 12:11:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 10 12:12:00.195048 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3f0832-2cc0-46fe-8396-9b0614a19294 None None] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:12:00.195048 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-1a3f0832-2cc0-46fe-8396-9b0614a19294 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:12:00.202477 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3f0832-2cc0-46fe-8396-9b0614a19294 None None] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:12:00.209409 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a9e1e1-3745-477e-bd77-c61d8f2bbc35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:12:00.210132 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a9e1e1-3745-477e-bd77-c61d8f2bbc35 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:12:00.210412 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a9e1e1-3745-477e-bd77-c61d8f2bbc35 admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:12:00.210723 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a9e1e1-3745-477e-bd77-c61d8f2bbc35 admin admin] PATH_INFO: `/v3/projects/11848252824f47d295158d07ce2305cf` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:12:00.212055 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a9e1e1-3745-477e-bd77-c61d8f2bbc35 admin admin] RBAC: Authorizing `identity:delete_project(project_id=11848252824f47d295158d07ce2305cf)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:12:00.214041 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a9e1e1-3745-477e-bd77-c61d8f2bbc35 admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:12:00.343546 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-06a9e1e1-3745-477e-bd77-c61d8f2bbc35 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '11848252824f47d295158d07ce2305cf', 'request_id': 'req-06a9e1e1-3745-477e-bd77-c61d8f2bbc35'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:12:00.350229 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-06a9e1e1-3745-477e-bd77-c61d8f2bbc35 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '11848252824f47d295158d07ce2305cf', 'request_id': 'req-06a9e1e1-3745-477e-bd77-c61d8f2bbc35'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:12:00.357422 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-06a9e1e1-3745-477e-bd77-c61d8f2bbc35 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '11848252824f47d295158d07ce2305cf', 'request_id': 'req-06a9e1e1-3745-477e-bd77-c61d8f2bbc35'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:12:00.451706 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-06a9e1e1-3745-477e-bd77-c61d8f2bbc35 admin admin] The token cache is being invalidate because project 11848252824f47d295158d07ce2305cf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81676) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 10 12:12:00.452019 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-06a9e1e1-3745-477e-bd77-c61d8f2bbc35 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:12:00.454422 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-06a9e1e1-3745-477e-bd77-c61d8f2bbc35 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:12:00.454740 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1099/2221] 10.222.0.46 () {58 vars in 1233 bytes} [Mon Oct 10 12:12:00 2022] DELETE /identity/v3/projects/11848252824f47d295158d07ce2305cf => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:12:00.460916 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511478a2-719b-46ef-9bc7-5d198d7fb9ef None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:12:00.462572 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511478a2-719b-46ef-9bc7-5d198d7fb9ef None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:12:00.463528 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511478a2-719b-46ef-9bc7-5d198d7fb9ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:12:00.501886 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-511478a2-719b-46ef-9bc7-5d198d7fb9ef None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:12:00.504195 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-511478a2-719b-46ef-9bc7-5d198d7fb9ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:12:00.530004 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:12:00.590494 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1123/2222] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:12:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:12:00.599247 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f033d7be-b5e6-43fa-8b21-6d5400507766 None None] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:12:00.599533 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f033d7be-b5e6-43fa-8b21-6d5400507766 None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:12:00.599992 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f033d7be-b5e6-43fa-8b21-6d5400507766 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:12:00.622644 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.auth.core [None req-f033d7be-b5e6-43fa-8b21-6d5400507766 None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:12:00.623919 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-f033d7be-b5e6-43fa-8b21-6d5400507766 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:12:00.633325 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1100/2223] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:12:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:12:00.637631 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511478a2-719b-46ef-9bc7-5d198d7fb9ef None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:12:00.647740 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511478a2-719b-46ef-9bc7-5d198d7fb9ef None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:12:00.652568 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ba2c46-d2f1-4ed9-a5a7-b04b7fbedd97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:12:00.653263 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ba2c46-d2f1-4ed9-a5a7-b04b7fbedd97 admin admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:12:00.653555 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ba2c46-d2f1-4ed9-a5a7-b04b7fbedd97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:12:00.653831 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ba2c46-d2f1-4ed9-a5a7-b04b7fbedd97 admin admin] PATH_INFO: `/v3/domains` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:12:00.654523 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ba2c46-d2f1-4ed9-a5a7-b04b7fbedd97 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 12:12:00.654848 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ba2c46-d2f1-4ed9-a5a7-b04b7fbedd97 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:12:00.656776 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ba2c46-d2f1-4ed9-a5a7-b04b7fbedd97 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:12:00.669905 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1124/2224] 10.222.0.46 () {58 vars in 1153 bytes} [Mon Oct 10 12:12:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 12:12:00.675041 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f033d7be-b5e6-43fa-8b21-6d5400507766 None None] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:12:00.689533 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f033d7be-b5e6-43fa-8b21-6d5400507766 None None] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:12:00.690924 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75086b0a-9ac4-40bc-ab5c-90912ed494cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:12:00.691535 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75086b0a-9ac4-40bc-ab5c-90912ed494cc admin admin] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:12:00.691819 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75086b0a-9ac4-40bc-ab5c-90912ed494cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:12:00.692093 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75086b0a-9ac4-40bc-ab5c-90912ed494cc admin admin] PATH_INFO: `/v3/projects` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:12:00.692727 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75086b0a-9ac4-40bc-ab5c-90912ed494cc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:12:00.694661 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75086b0a-9ac4-40bc-ab5c-90912ed494cc admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:12:00.783568 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:12:00.793675 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1101/2225] 10.222.0.46 () {60 vars in 1153 bytes} [Mon Oct 10 12:12:00 2022] POST /identity/v3/projects => generated 365 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 12:12:00.794850 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ba2c46-d2f1-4ed9-a5a7-b04b7fbedd97 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:12:00.805802 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ba2c46-d2f1-4ed9-a5a7-b04b7fbedd97 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:12:00.869942 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abda9e25-3023-46ba-89a4-17bfa86dea4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:12:00.870567 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abda9e25-3023-46ba-89a4-17bfa86dea4e admin admin] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:12:00.871020 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abda9e25-3023-46ba-89a4-17bfa86dea4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:12:00.871400 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abda9e25-3023-46ba-89a4-17bfa86dea4e admin admin] PATH_INFO: `/v3/users` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:12:00.871835 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abda9e25-3023-46ba-89a4-17bfa86dea4e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:12:00.873767 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abda9e25-3023-46ba-89a4-17bfa86dea4e admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:12:01.005379 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1125/2226] 10.222.0.46 () {60 vars in 1144 bytes} [Mon Oct 10 12:12:00 2022] POST /identity/v3/users => generated 283 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 10 12:12:01.009988 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75086b0a-9ac4-40bc-ab5c-90912ed494cc admin admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:12:01.026062 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75086b0a-9ac4-40bc-ab5c-90912ed494cc admin admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:12:01.030911 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153d75d4-2a9d-4742-943e-b5e9df32aaaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:12:01.031531 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153d75d4-2a9d-4742-943e-b5e9df32aaaf admin admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:12:01.032302 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153d75d4-2a9d-4742-943e-b5e9df32aaaf admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:12:01.032578 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153d75d4-2a9d-4742-943e-b5e9df32aaaf admin admin] PATH_INFO: `/v3/roles` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:12:01.033240 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153d75d4-2a9d-4742-943e-b5e9df32aaaf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:12:01.035055 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153d75d4-2a9d-4742-943e-b5e9df32aaaf admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:12:01.049992 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1102/2227] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 10 12:12:01 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 12:12:01.055630 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abda9e25-3023-46ba-89a4-17bfa86dea4e admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:12:01.066498 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abda9e25-3023-46ba-89a4-17bfa86dea4e admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:12:01.071480 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d0233e-88a2-4e97-84cb-db2e1d5e1f5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:12:01.072514 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d0233e-88a2-4e97-84cb-db2e1d5e1f5e admin admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:12:01.072514 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d0233e-88a2-4e97-84cb-db2e1d5e1f5e admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:12:01.072745 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d0233e-88a2-4e97-84cb-db2e1d5e1f5e admin admin] PATH_INFO: `/v3/roles` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:12:01.073293 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d0233e-88a2-4e97-84cb-db2e1d5e1f5e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:12:01.074939 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d0233e-88a2-4e97-84cb-db2e1d5e1f5e admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:12:01.088152 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1126/2228] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 10 12:12:01 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 12:12:01.092104 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153d75d4-2a9d-4742-943e-b5e9df32aaaf admin admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:12:01.103927 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153d75d4-2a9d-4742-943e-b5e9df32aaaf admin admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:12:01.109757 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea1b7d6-97f2-42fc-b6ab-8385d352b798 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:12:01.110968 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea1b7d6-97f2-42fc-b6ab-8385d352b798 admin admin] REQUEST_METHOD: `PUT` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:12:01.116639 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea1b7d6-97f2-42fc-b6ab-8385d352b798 admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:12:01.116639 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea1b7d6-97f2-42fc-b6ab-8385d352b798 admin admin] PATH_INFO: `/v3/projects/94a3ca0d7d854e26a5e26ee48de284c4/users/ce63f9c620d34573b330130578f6dc44/roles/b2cb2a6e818e47338ce071e6fedfc673` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:12:01.130648 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea1b7d6-97f2-42fc-b6ab-8385d352b798 admin admin] RBAC: Authorizing `identity:create_grant(project_id=94a3ca0d7d854e26a5e26ee48de284c4, user_id=ce63f9c620d34573b330130578f6dc44, role_id=b2cb2a6e818e47338ce071e6fedfc673)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:12:01.133585 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea1b7d6-97f2-42fc-b6ab-8385d352b798 admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:12:01.210729 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1103/2229] 10.222.0.46 () {60 vars in 1483 bytes} [Mon Oct 10 12:12:01 2022] PUT /identity/v3/projects/94a3ca0d7d854e26a5e26ee48de284c4/users/ce63f9c620d34573b330130578f6dc44/roles/b2cb2a6e818e47338ce071e6fedfc673 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 10 12:12:01.222785 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e174e2-1051-4eb4-82c9-2cd3b5af8763 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:12:01.222785 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e174e2-1051-4eb4-82c9-2cd3b5af8763 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:12:01.222785 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e174e2-1051-4eb4-82c9-2cd3b5af8763 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:12:01.253704 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-39e174e2-1051-4eb4-82c9-2cd3b5af8763 None None] MFA Rules not processed for user `ce63f9c620d34573b330130578f6dc44`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:12:01.254296 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-39e174e2-1051-4eb4-82c9-2cd3b5af8763 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:12:01.331034 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1127/2230] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:12:01 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 10 12:12:01.343167 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea1b7d6-97f2-42fc-b6ab-8385d352b798 admin admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:12:01.345766 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-5ea1b7d6-97f2-42fc-b6ab-8385d352b798 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:12:01.410963 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea1b7d6-97f2-42fc-b6ab-8385d352b798 admin admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:12:01.414996 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454e3415-7d77-4b1e-b26d-4e2938c10085 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3ae093470484b93b23dcfae1657df9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad3772f7b6a47c28b7ba37a9316a3c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:12:01.415618 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454e3415-7d77-4b1e-b26d-4e2938c10085 service cinder] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:12:01.415901 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454e3415-7d77-4b1e-b26d-4e2938c10085 service cinder] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:12:01.416175 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454e3415-7d77-4b1e-b26d-4e2938c10085 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:12:01.425900 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454e3415-7d77-4b1e-b26d-4e2938c10085 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:12:01.427428 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454e3415-7d77-4b1e-b26d-4e2938c10085 service cinder] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:12:01.437361 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1104/2231] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 10 12:12:01 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:12:11.326790 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=89,keystone:INSERT=3 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:12:11.432906 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=41,keystone:INSERT=1 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:12:17.952769 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c969233a-4591-476c-892e-b89ebb95b194 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:12:17.953271 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c969233a-4591-476c-892e-b89ebb95b194 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:12:17.953448 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c969233a-4591-476c-892e-b89ebb95b194 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:12:17.974047 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-c969233a-4591-476c-892e-b89ebb95b194 None None] MFA Rules not processed for user `d6430593eecf45d8b891c3b55de4a944`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:12:17.976409 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-c969233a-4591-476c-892e-b89ebb95b194 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:12:18.030736 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1128/2232] 10.222.0.46 () {58 vars in 1009 bytes} [Mon Oct 10 12:12:17 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:12:18.076112 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454e3415-7d77-4b1e-b26d-4e2938c10085 service cinder] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:12:18.077588 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-454e3415-7d77-4b1e-b26d-4e2938c10085 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:12:18.147441 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454e3415-7d77-4b1e-b26d-4e2938c10085 service cinder] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:12:18.152447 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c3feaa-e541-499e-9ea3-7d06752c7ade service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15ef3d53ea484237adb0b3d64fdc5480', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad3772f7b6a47c28b7ba37a9316a3c1', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:12:18.153051 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c3feaa-e541-499e-9ea3-7d06752c7ade service swift] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:12:18.153333 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c3feaa-e541-499e-9ea3-7d06752c7ade service swift] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:12:18.153609 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c3feaa-e541-499e-9ea3-7d06752c7ade service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:12:18.162929 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c3feaa-e541-499e-9ea3-7d06752c7ade service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:12:18.164515 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c3feaa-e541-499e-9ea3-7d06752c7ade service swift] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:12:18.172742 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1105/2233] 10.222.0.46 () {60 vars in 1384 bytes} [Mon Oct 10 12:12:18 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 10 12:12:22.175368 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa1b9f1-1ff3-431c-9121-2a22a848bc47 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:12:22.175906 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa1b9f1-1ff3-431c-9121-2a22a848bc47 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:12:22.175906 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa1b9f1-1ff3-431c-9121-2a22a848bc47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:12:22.208969 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-cfa1b9f1-1ff3-431c-9121-2a22a848bc47 None None] MFA Rules not processed for user `d6430593eecf45d8b891c3b55de4a944`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:12:22.208969 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-cfa1b9f1-1ff3-431c-9121-2a22a848bc47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:12:22.212494 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1129/2234] 10.222.0.46 () {58 vars in 1009 bytes} [Mon Oct 10 12:12:22 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:12:22.229531 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c3feaa-e541-499e-9ea3-7d06752c7ade service swift] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:12:22.252768 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c3feaa-e541-499e-9ea3-7d06752c7ade service swift] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:12:22.263542 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513eab7f-2ae5-439f-9dd2-41f126d2c409 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15ef3d53ea484237adb0b3d64fdc5480', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad3772f7b6a47c28b7ba37a9316a3c1', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:12:22.264849 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513eab7f-2ae5-439f-9dd2-41f126d2c409 service swift] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:12:22.265425 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513eab7f-2ae5-439f-9dd2-41f126d2c409 service swift] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:12:22.265921 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513eab7f-2ae5-439f-9dd2-41f126d2c409 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:12:22.288193 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513eab7f-2ae5-439f-9dd2-41f126d2c409 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:12:22.293719 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513eab7f-2ae5-439f-9dd2-41f126d2c409 service swift] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:12:22.355592 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1106/2235] 10.222.0.46 () {60 vars in 1384 bytes} [Mon Oct 10 12:12:22 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 10 12:12:32.193554 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=28 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:12:32.341277 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=38 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:12:38.039909 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa1b9f1-1ff3-431c-9121-2a22a848bc47 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:12:38.043244 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-cfa1b9f1-1ff3-431c-9121-2a22a848bc47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:12:38.180012 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa1b9f1-1ff3-431c-9121-2a22a848bc47 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:12:38.184567 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00cfcc74-80d7-47d8-8073-2a10d1596444 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58b1c7c85302421cba823ace4201861f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad3772f7b6a47c28b7ba37a9316a3c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:12:38.185264 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cfcc74-80d7-47d8-8073-2a10d1596444 service neutron] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:12:38.185565 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cfcc74-80d7-47d8-8073-2a10d1596444 service neutron] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:12:38.186022 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cfcc74-80d7-47d8-8073-2a10d1596444 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:12:38.187728 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-00cfcc74-80d7-47d8-8073-2a10d1596444 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:12:38.294153 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00cfcc74-80d7-47d8-8073-2a10d1596444 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:12:38.295826 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00cfcc74-80d7-47d8-8073-2a10d1596444 service neutron] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:12:38.316608 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1130/2236] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 10 12:12:38 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 278 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:12:39.520553 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513eab7f-2ae5-439f-9dd2-41f126d2c409 service swift] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:12:39.531475 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513eab7f-2ae5-439f-9dd2-41f126d2c409 service swift] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:12:39.536215 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7746597-9a2f-4254-982d-51ba8ba2beb6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58b1c7c85302421cba823ace4201861f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad3772f7b6a47c28b7ba37a9316a3c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:12:39.537105 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7746597-9a2f-4254-982d-51ba8ba2beb6 service neutron] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:12:39.537522 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7746597-9a2f-4254-982d-51ba8ba2beb6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:12:39.537872 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7746597-9a2f-4254-982d-51ba8ba2beb6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:12:39.540361 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-a7746597-9a2f-4254-982d-51ba8ba2beb6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:12:39.560911 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7746597-9a2f-4254-982d-51ba8ba2beb6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:12:39.562459 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7746597-9a2f-4254-982d-51ba8ba2beb6 service neutron] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:12:39.575649 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1107/2237] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 10 12:12:39 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:12:43.339400 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94321118-48f9-4890-9372-dc92183c2a88 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:12:43.339910 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94321118-48f9-4890-9372-dc92183c2a88 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:12:43.340007 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94321118-48f9-4890-9372-dc92183c2a88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:12:43.359246 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-94321118-48f9-4890-9372-dc92183c2a88 None None] MFA Rules not processed for user `d6430593eecf45d8b891c3b55de4a944`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:12:43.360449 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-94321118-48f9-4890-9372-dc92183c2a88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:12:43.367359 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1131/2238] 10.222.0.46 () {58 vars in 967 bytes} [Mon Oct 10 12:12:43 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 10 12:12:43.380708 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7746597-9a2f-4254-982d-51ba8ba2beb6 service neutron] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:12:43.392689 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7746597-9a2f-4254-982d-51ba8ba2beb6 service neutron] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:12:43.397640 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402c1187-f5da-4f56-95a0-dbc215fa9fca service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15ef3d53ea484237adb0b3d64fdc5480', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad3772f7b6a47c28b7ba37a9316a3c1', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:12:43.398269 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402c1187-f5da-4f56-95a0-dbc215fa9fca service swift] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:12:43.398541 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402c1187-f5da-4f56-95a0-dbc215fa9fca service swift] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:12:43.398853 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402c1187-f5da-4f56-95a0-dbc215fa9fca service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:12:43.407514 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402c1187-f5da-4f56-95a0-dbc215fa9fca service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:12:43.409077 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402c1187-f5da-4f56-95a0-dbc215fa9fca service swift] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:12:43.417051 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1108/2239] 10.222.0.46 () {60 vars in 1384 bytes} [Mon Oct 10 12:12:43 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 10 12:12:50.848474 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b477f175-dde9-40ac-a658-92c4ddcd2bd1 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:12:50.853456 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b477f175-dde9-40ac-a658-92c4ddcd2bd1 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:12:50.853456 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b477f175-dde9-40ac-a658-92c4ddcd2bd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:12:50.870262 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-b477f175-dde9-40ac-a658-92c4ddcd2bd1 None None] MFA Rules not processed for user `13ac3580c0164d20994d68744d9153b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:12:50.871565 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-b477f175-dde9-40ac-a658-92c4ddcd2bd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:12:50.936293 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1132/2240] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:12:50 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:12:51.655611 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402c1187-f5da-4f56-95a0-dbc215fa9fca service swift] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:12:51.657264 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-402c1187-f5da-4f56-95a0-dbc215fa9fca service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:12:51.828934 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402c1187-f5da-4f56-95a0-dbc215fa9fca service swift] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:12:51.842602 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6bf0fc5-41b8-4fee-85c5-07c0513a9510 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:12:51.844150 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bf0fc5-41b8-4fee-85c5-07c0513a9510 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:12:51.846125 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bf0fc5-41b8-4fee-85c5-07c0513a9510 admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:12:51.849060 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bf0fc5-41b8-4fee-85c5-07c0513a9510 admin admin] PATH_INFO: `/v3/users/d58683f7689a4356badd8f4bb30850a5` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:12:51.851978 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6bf0fc5-41b8-4fee-85c5-07c0513a9510 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d58683f7689a4356badd8f4bb30850a5)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:12:51.856089 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6bf0fc5-41b8-4fee-85c5-07c0513a9510 admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:12:52.105512 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-d6bf0fc5-41b8-4fee-85c5-07c0513a9510 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd58683f7689a4356badd8f4bb30850a5'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:12:52.117530 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-d6bf0fc5-41b8-4fee-85c5-07c0513a9510 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd58683f7689a4356badd8f4bb30850a5'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:12:52.118716 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-d6bf0fc5-41b8-4fee-85c5-07c0513a9510 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd58683f7689a4356badd8f4bb30850a5'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:12:52.208612 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-d6bf0fc5-41b8-4fee-85c5-07c0513a9510 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd58683f7689a4356badd8f4bb30850a5'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:12:52.225014 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-d6bf0fc5-41b8-4fee-85c5-07c0513a9510 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd58683f7689a4356badd8f4bb30850a5'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:12:52.233269 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1109/2241] 10.222.0.46 () {58 vars in 1224 bytes} [Mon Oct 10 12:12:51 2022] DELETE /identity/v3/users/d58683f7689a4356badd8f4bb30850a5 => generated 0 bytes in 578 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 10 12:12:52.530801 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b477f175-dde9-40ac-a658-92c4ddcd2bd1 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:12:52.543460 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b477f175-dde9-40ac-a658-92c4ddcd2bd1 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:12:52.595025 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:12:52.613552 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9836ef-eac2-438b-9812-75b5ce39cabf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:12:52.614194 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9836ef-eac2-438b-9812-75b5ce39cabf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:12:52.614477 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9836ef-eac2-438b-9812-75b5ce39cabf admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:12:52.614797 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9836ef-eac2-438b-9812-75b5ce39cabf admin admin] PATH_INFO: `/v3/projects/4326aa9c66954652943d3754b121bce8` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:12:52.615924 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9836ef-eac2-438b-9812-75b5ce39cabf admin admin] RBAC: Authorizing `identity:delete_project(project_id=4326aa9c66954652943d3754b121bce8)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:12:52.617952 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9836ef-eac2-438b-9812-75b5ce39cabf admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:12:52.781172 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-0c9836ef-eac2-438b-9812-75b5ce39cabf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4326aa9c66954652943d3754b121bce8', 'request_id': 'req-0c9836ef-eac2-438b-9812-75b5ce39cabf'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:12:52.786912 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-0c9836ef-eac2-438b-9812-75b5ce39cabf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4326aa9c66954652943d3754b121bce8', 'request_id': 'req-0c9836ef-eac2-438b-9812-75b5ce39cabf'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:12:52.793984 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-0c9836ef-eac2-438b-9812-75b5ce39cabf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4326aa9c66954652943d3754b121bce8', 'request_id': 'req-0c9836ef-eac2-438b-9812-75b5ce39cabf'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:12:52.929801 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-0c9836ef-eac2-438b-9812-75b5ce39cabf admin admin] The token cache is being invalidate because project 4326aa9c66954652943d3754b121bce8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81677) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 10 12:12:52.933112 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-0c9836ef-eac2-438b-9812-75b5ce39cabf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:12:52.933112 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-0c9836ef-eac2-438b-9812-75b5ce39cabf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:12:52.933112 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1133/2242] 10.222.0.46 () {58 vars in 1233 bytes} [Mon Oct 10 12:12:52 2022] DELETE /identity/v3/projects/4326aa9c66954652943d3754b121bce8 => generated 0 bytes in 402 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:12:56.752626 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6bf0fc5-41b8-4fee-85c5-07c0513a9510 admin admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:12:56.754714 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-d6bf0fc5-41b8-4fee-85c5-07c0513a9510 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:12:56.815696 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:12:56.842080 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6bf0fc5-41b8-4fee-85c5-07c0513a9510 admin admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:12:56.905953 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337b644e-8863-425f-8b4d-a0ad249ea9a4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f130bbb69e546ce99b4e14de81d7e54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad3772f7b6a47c28b7ba37a9316a3c1', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:12:56.906564 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337b644e-8863-425f-8b4d-a0ad249ea9a4 service nova] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:12:56.906878 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337b644e-8863-425f-8b4d-a0ad249ea9a4 service nova] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:12:56.907162 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337b644e-8863-425f-8b4d-a0ad249ea9a4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:12:56.908911 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-337b644e-8863-425f-8b4d-a0ad249ea9a4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:12:56.967063 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337b644e-8863-425f-8b4d-a0ad249ea9a4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:12:56.968663 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337b644e-8863-425f-8b4d-a0ad249ea9a4 service nova] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:12:56.978035 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1110/2243] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 10 12:12:56 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 227 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:12:58.004260 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9836ef-eac2-438b-9812-75b5ce39cabf admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:12:58.005491 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-0c9836ef-eac2-438b-9812-75b5ce39cabf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:12:58.079055 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9836ef-eac2-438b-9812-75b5ce39cabf admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:12:58.083747 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4d7b4a-257a-401f-bd25-5861e4a31936 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58b1c7c85302421cba823ace4201861f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad3772f7b6a47c28b7ba37a9316a3c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:12:58.084375 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4d7b4a-257a-401f-bd25-5861e4a31936 service neutron] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:12:58.084652 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4d7b4a-257a-401f-bd25-5861e4a31936 service neutron] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:12:58.085435 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4d7b4a-257a-401f-bd25-5861e4a31936 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:12:58.087543 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-fb4d7b4a-257a-401f-bd25-5861e4a31936 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:12:58.155811 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4d7b4a-257a-401f-bd25-5861e4a31936 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:12:58.157958 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4d7b4a-257a-401f-bd25-5861e4a31936 service neutron] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:12:58.170476 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1134/2244] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 10 12:12:58 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:12:59.663806 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337b644e-8863-425f-8b4d-a0ad249ea9a4 service nova] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:12:59.666002 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-337b644e-8863-425f-8b4d-a0ad249ea9a4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:12:59.681077 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337b644e-8863-425f-8b4d-a0ad249ea9a4 service nova] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:12:59.687235 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6eec644-9d92-44c1-be8b-28db72707ef2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f130bbb69e546ce99b4e14de81d7e54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad3772f7b6a47c28b7ba37a9316a3c1', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:12:59.687853 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6eec644-9d92-44c1-be8b-28db72707ef2 service nova] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:12:59.688180 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6eec644-9d92-44c1-be8b-28db72707ef2 service nova] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:12:59.688645 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6eec644-9d92-44c1-be8b-28db72707ef2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:12:59.690753 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-a6eec644-9d92-44c1-be8b-28db72707ef2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:12:59.761325 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6eec644-9d92-44c1-be8b-28db72707ef2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:12:59.763332 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6eec644-9d92-44c1-be8b-28db72707ef2 service nova] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:12:59.775443 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1111/2245] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 10 12:12:59 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:13:01.106778 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ebc1f2-d486-4c82-98ef-920c5ac326f9 None None] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:13:01.107725 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ebc1f2-d486-4c82-98ef-920c5ac326f9 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:13:01.108022 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ebc1f2-d486-4c82-98ef-920c5ac326f9 None None] PATH_INFO: `/` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:13:01.109697 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1135/2246] 10.222.0.46 () {54 vars in 921 bytes} [Mon Oct 10 12:13:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 12:13:01.122772 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dab6e04-66d4-410f-9391-2e49371a0bbf None None] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:13:01.124029 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dab6e04-66d4-410f-9391-2e49371a0bbf None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:13:01.124465 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dab6e04-66d4-410f-9391-2e49371a0bbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:13:01.174391 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.auth.core [None req-5dab6e04-66d4-410f-9391-2e49371a0bbf None None] MFA Rules not processed for user `5f130bbb69e546ce99b4e14de81d7e54`. Rule list: `[]` (Enabled: `True`). {{(pid=81676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:13:01.174391 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-5dab6e04-66d4-410f-9391-2e49371a0bbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:13:01.198050 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1112/2247] 10.222.0.46 () {58 vars in 1018 bytes} [Mon Oct 10 12:13:01 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:13:01.212421 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ebc1f2-d486-4c82-98ef-920c5ac326f9 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:13:01.214516 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-e5ebc1f2-d486-4c82-98ef-920c5ac326f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:13:01.242353 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:13:01.317599 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ebc1f2-d486-4c82-98ef-920c5ac326f9 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:13:01.322134 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1d35b8-7d7a-42b3-a490-9df3a1326f8e service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '124f4aa8ab244ca2aadf4f2614df51e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad3772f7b6a47c28b7ba37a9316a3c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:13:01.322926 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1d35b8-7d7a-42b3-a490-9df3a1326f8e service glance] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:13:01.327509 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1d35b8-7d7a-42b3-a490-9df3a1326f8e service glance] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:13:01.327509 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1d35b8-7d7a-42b3-a490-9df3a1326f8e service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:13:01.337529 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1d35b8-7d7a-42b3-a490-9df3a1326f8e service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:13:01.339131 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1d35b8-7d7a-42b3-a490-9df3a1326f8e service glance] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:13:01.353582 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1136/2248] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 10 12:13:01 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:13:01.583373 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2458787-1f6a-4b54-b263-c6fcf60f8e8e None None] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:13:01.583847 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2458787-1f6a-4b54-b263-c6fcf60f8e8e None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:13:01.584179 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2458787-1f6a-4b54-b263-c6fcf60f8e8e None None] PATH_INFO: `/` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:13:01.586132 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1113/2249] 10.222.0.46 () {54 vars in 921 bytes} [Mon Oct 10 12:13:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 12:13:01.592344 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e96399-9806-403d-a702-7e6545be16a6 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:13:01.592558 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e96399-9806-403d-a702-7e6545be16a6 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:13:01.593090 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e96399-9806-403d-a702-7e6545be16a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:13:01.616286 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-53e96399-9806-403d-a702-7e6545be16a6 None None] MFA Rules not processed for user `58b1c7c85302421cba823ace4201861f`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:13:01.617856 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-53e96399-9806-403d-a702-7e6545be16a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:13:01.625602 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1137/2250] 10.222.0.46 () {58 vars in 1018 bytes} [Mon Oct 10 12:13:01 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:13:01.644066 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2458787-1f6a-4b54-b263-c6fcf60f8e8e None None] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:13:01.662958 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2458787-1f6a-4b54-b263-c6fcf60f8e8e None None] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:13:01.667992 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0b88fb-8a92-41f0-a659-d6dc14e958bb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58b1c7c85302421cba823ace4201861f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad3772f7b6a47c28b7ba37a9316a3c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:13:01.669135 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0b88fb-8a92-41f0-a659-d6dc14e958bb service neutron] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:13:01.671193 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0b88fb-8a92-41f0-a659-d6dc14e958bb service neutron] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:13:01.676828 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0b88fb-8a92-41f0-a659-d6dc14e958bb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:13:01.680541 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0b88fb-8a92-41f0-a659-d6dc14e958bb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:13:01.682125 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0b88fb-8a92-41f0-a659-d6dc14e958bb service neutron] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:13:01.691313 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1114/2251] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 10 12:13:01 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:13:01.807026 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62798b49-c062-44b4-961f-e0c11a78f006 None None] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:13:01.807308 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62798b49-c062-44b4-961f-e0c11a78f006 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:13:01.807584 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62798b49-c062-44b4-961f-e0c11a78f006 None None] PATH_INFO: `/` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:13:01.808931 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1138/2252] 10.222.0.46 () {54 vars in 921 bytes} [Mon Oct 10 12:13:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 10 12:13:01.817083 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818cbcf7-eef9-4ced-8517-b0560501815f None None] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:13:01.819667 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818cbcf7-eef9-4ced-8517-b0560501815f None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:13:01.819667 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818cbcf7-eef9-4ced-8517-b0560501815f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:13:01.836148 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:13:01.842866 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.auth.core [None req-818cbcf7-eef9-4ced-8517-b0560501815f None None] MFA Rules not processed for user `5f130bbb69e546ce99b4e14de81d7e54`. Rule list: `[]` (Enabled: `True`). {{(pid=81676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:13:01.844017 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-818cbcf7-eef9-4ced-8517-b0560501815f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:13:01.857144 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1115/2253] 10.222.0.46 () {58 vars in 1018 bytes} [Mon Oct 10 12:13:01 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:13:01.870249 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62798b49-c062-44b4-961f-e0c11a78f006 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:13:01.883165 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62798b49-c062-44b4-961f-e0c11a78f006 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:13:01.887040 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5c641f-c6a2-4b84-9297-21844b6dc2b5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58b1c7c85302421cba823ace4201861f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad3772f7b6a47c28b7ba37a9316a3c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:13:01.887651 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5c641f-c6a2-4b84-9297-21844b6dc2b5 service neutron] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:13:01.887924 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5c641f-c6a2-4b84-9297-21844b6dc2b5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:13:01.888195 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5c641f-c6a2-4b84-9297-21844b6dc2b5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:13:01.899192 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5c641f-c6a2-4b84-9297-21844b6dc2b5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:13:01.905683 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5c641f-c6a2-4b84-9297-21844b6dc2b5 service neutron] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:13:01.914477 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1139/2254] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 10 12:13:01 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:13:02.177067 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818cbcf7-eef9-4ced-8517-b0560501815f None None] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:13:02.178602 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-818cbcf7-eef9-4ced-8517-b0560501815f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:13:02.276525 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818cbcf7-eef9-4ced-8517-b0560501815f None None] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:13:02.283879 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b1ff2b-28c7-488e-9d7f-cb970da69b91 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e8f9bb1776d47169ef7febb57c954cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad3772f7b6a47c28b7ba37a9316a3c1', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:13:02.284619 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b1ff2b-28c7-488e-9d7f-cb970da69b91 service placement] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:13:02.284907 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b1ff2b-28c7-488e-9d7f-cb970da69b91 service placement] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:13:02.285191 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b1ff2b-28c7-488e-9d7f-cb970da69b91 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:13:02.287765 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-15b1ff2b-28c7-488e-9d7f-cb970da69b91 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:13:02.304942 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b1ff2b-28c7-488e-9d7f-cb970da69b91 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:13:02.306746 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b1ff2b-28c7-488e-9d7f-cb970da69b91 service placement] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:13:02.321967 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1116/2255] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 10 12:13:02 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:13:02.351560 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5c641f-c6a2-4b84-9297-21844b6dc2b5 service neutron] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:13:02.353645 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-ed5c641f-c6a2-4b84-9297-21844b6dc2b5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:13:02.381943 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5c641f-c6a2-4b84-9297-21844b6dc2b5 service neutron] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:13:02.388239 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d819e5-13c2-4142-9d81-9ef139b7a2ea service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f130bbb69e546ce99b4e14de81d7e54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad3772f7b6a47c28b7ba37a9316a3c1', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:13:02.389147 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d819e5-13c2-4142-9d81-9ef139b7a2ea service nova] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:13:02.389443 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d819e5-13c2-4142-9d81-9ef139b7a2ea service nova] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:13:02.389745 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d819e5-13c2-4142-9d81-9ef139b7a2ea service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:13:02.391663 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-d9d819e5-13c2-4142-9d81-9ef139b7a2ea service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:13:02.507142 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d819e5-13c2-4142-9d81-9ef139b7a2ea service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:13:02.509609 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d819e5-13c2-4142-9d81-9ef139b7a2ea service nova] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:13:02.523041 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1140/2256] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 10 12:13:02 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:13:12.314130 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=32 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:13:12.518226 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=54 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:13:13.655724 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f698538-5a67-449e-84e5-8ae7f15da750 None None] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:13:13.656380 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f698538-5a67-449e-84e5-8ae7f15da750 None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:13:13.656380 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f698538-5a67-449e-84e5-8ae7f15da750 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:13:13.678786 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.auth.core [None req-0f698538-5a67-449e-84e5-8ae7f15da750 None None] MFA Rules not processed for user `d6430593eecf45d8b891c3b55de4a944`. Rule list: `[]` (Enabled: `True`). {{(pid=81676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:13:13.679958 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-0f698538-5a67-449e-84e5-8ae7f15da750 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:13:13.749467 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1117/2257] 10.222.0.46 () {58 vars in 1009 bytes} [Mon Oct 10 12:13:13 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:13:13.775781 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d819e5-13c2-4142-9d81-9ef139b7a2ea service nova] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:13:13.775781 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-d9d819e5-13c2-4142-9d81-9ef139b7a2ea service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:13:13.849004 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d819e5-13c2-4142-9d81-9ef139b7a2ea service nova] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:13:13.854063 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdebb7fd-58ba-4ed1-a4f6-7ec885971f45 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15ef3d53ea484237adb0b3d64fdc5480', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad3772f7b6a47c28b7ba37a9316a3c1', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:13:13.854771 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdebb7fd-58ba-4ed1-a4f6-7ec885971f45 service swift] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:13:13.855331 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdebb7fd-58ba-4ed1-a4f6-7ec885971f45 service swift] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:13:13.855331 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdebb7fd-58ba-4ed1-a4f6-7ec885971f45 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:13:13.864430 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdebb7fd-58ba-4ed1-a4f6-7ec885971f45 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:13:13.866085 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdebb7fd-58ba-4ed1-a4f6-7ec885971f45 service swift] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:13:13.874865 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1141/2258] 10.222.0.46 () {60 vars in 1384 bytes} [Mon Oct 10 12:13:13 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 10 12:13:23.739914 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=25 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:13:23.871304 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=32 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:13:39.156692 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f698538-5a67-449e-84e5-8ae7f15da750 None None] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:13:39.166490 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f698538-5a67-449e-84e5-8ae7f15da750 None None] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:13:39.170780 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778c49c7-915e-4582-871e-9696efca0989 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58b1c7c85302421cba823ace4201861f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad3772f7b6a47c28b7ba37a9316a3c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:13:39.172491 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778c49c7-915e-4582-871e-9696efca0989 service neutron] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:13:39.172491 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778c49c7-915e-4582-871e-9696efca0989 service neutron] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:13:39.172491 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778c49c7-915e-4582-871e-9696efca0989 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:13:39.173726 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-778c49c7-915e-4582-871e-9696efca0989 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:13:39.183789 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778c49c7-915e-4582-871e-9696efca0989 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:13:39.185206 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778c49c7-915e-4582-871e-9696efca0989 service neutron] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:13:39.193084 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1118/2259] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 10 12:13:39 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:13:41.548211 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdebb7fd-58ba-4ed1-a4f6-7ec885971f45 service swift] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:13:41.549669 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-fdebb7fd-58ba-4ed1-a4f6-7ec885971f45 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:13:41.614893 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdebb7fd-58ba-4ed1-a4f6-7ec885971f45 service swift] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:13:41.619922 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef2f61b-627a-4e32-b6d3-42fc4f456cb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:13:41.620543 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef2f61b-627a-4e32-b6d3-42fc4f456cb6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:13:41.620823 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef2f61b-627a-4e32-b6d3-42fc4f456cb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:13:41.621098 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef2f61b-627a-4e32-b6d3-42fc4f456cb6 admin admin] PATH_INFO: `/v3/users/ce63f9c620d34573b330130578f6dc44` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:13:41.622199 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef2f61b-627a-4e32-b6d3-42fc4f456cb6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ce63f9c620d34573b330130578f6dc44)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:13:41.624148 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef2f61b-627a-4e32-b6d3-42fc4f456cb6 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:13:41.849775 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-4ef2f61b-627a-4e32-b6d3-42fc4f456cb6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ce63f9c620d34573b330130578f6dc44'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:13:41.857414 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-4ef2f61b-627a-4e32-b6d3-42fc4f456cb6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ce63f9c620d34573b330130578f6dc44'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:13:41.858086 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-4ef2f61b-627a-4e32-b6d3-42fc4f456cb6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ce63f9c620d34573b330130578f6dc44'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:13:41.915881 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-4ef2f61b-627a-4e32-b6d3-42fc4f456cb6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ce63f9c620d34573b330130578f6dc44'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:13:41.928991 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-4ef2f61b-627a-4e32-b6d3-42fc4f456cb6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ce63f9c620d34573b330130578f6dc44'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:13:41.935540 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1142/2260] 10.222.0.46 () {58 vars in 1224 bytes} [Mon Oct 10 12:13:41 2022] DELETE /identity/v3/users/ce63f9c620d34573b330130578f6dc44 => generated 0 bytes in 388 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:13:41.940144 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e88464e-112a-4ff2-95f3-ba139a2d5d2b None None] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:13:41.940659 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e88464e-112a-4ff2-95f3-ba139a2d5d2b None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:13:41.940756 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e88464e-112a-4ff2-95f3-ba139a2d5d2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:13:41.959302 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.auth.core [None req-9e88464e-112a-4ff2-95f3-ba139a2d5d2b None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:13:41.960490 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-9e88464e-112a-4ff2-95f3-ba139a2d5d2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:13:42.020931 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1119/2261] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:13:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:13:42.030762 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef2f61b-627a-4e32-b6d3-42fc4f456cb6 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:13:42.032226 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-4ef2f61b-627a-4e32-b6d3-42fc4f456cb6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:13:42.092839 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef2f61b-627a-4e32-b6d3-42fc4f456cb6 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:13:42.096790 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aefc04f-72f0-44bd-a235-ecf700c03562 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58b1c7c85302421cba823ace4201861f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad3772f7b6a47c28b7ba37a9316a3c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:13:42.097431 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aefc04f-72f0-44bd-a235-ecf700c03562 service neutron] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:13:42.097715 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aefc04f-72f0-44bd-a235-ecf700c03562 service neutron] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:13:42.097994 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aefc04f-72f0-44bd-a235-ecf700c03562 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:13:42.108677 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aefc04f-72f0-44bd-a235-ecf700c03562 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:13:42.110031 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aefc04f-72f0-44bd-a235-ecf700c03562 service neutron] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:13:42.120101 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1143/2262] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 10 12:13:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:13:42.575595 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e88464e-112a-4ff2-95f3-ba139a2d5d2b None None] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:13:42.577324 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-9e88464e-112a-4ff2-95f3-ba139a2d5d2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:13:42.591899 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e88464e-112a-4ff2-95f3-ba139a2d5d2b None None] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:13:42.597308 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dedd956-5128-4773-ad0a-f714cc62ea79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:13:42.597990 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dedd956-5128-4773-ad0a-f714cc62ea79 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:13:42.598262 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dedd956-5128-4773-ad0a-f714cc62ea79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:13:42.598532 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dedd956-5128-4773-ad0a-f714cc62ea79 admin admin] PATH_INFO: `/v3/projects/94a3ca0d7d854e26a5e26ee48de284c4` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:13:42.599717 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dedd956-5128-4773-ad0a-f714cc62ea79 admin admin] RBAC: Authorizing `identity:delete_project(project_id=94a3ca0d7d854e26a5e26ee48de284c4)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:13:42.601656 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dedd956-5128-4773-ad0a-f714cc62ea79 admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:13:42.715222 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-5dedd956-5128-4773-ad0a-f714cc62ea79 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '94a3ca0d7d854e26a5e26ee48de284c4', 'request_id': 'req-5dedd956-5128-4773-ad0a-f714cc62ea79'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:13:42.722053 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-5dedd956-5128-4773-ad0a-f714cc62ea79 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '94a3ca0d7d854e26a5e26ee48de284c4', 'request_id': 'req-5dedd956-5128-4773-ad0a-f714cc62ea79'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:13:42.730576 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-5dedd956-5128-4773-ad0a-f714cc62ea79 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '94a3ca0d7d854e26a5e26ee48de284c4', 'request_id': 'req-5dedd956-5128-4773-ad0a-f714cc62ea79'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:13:42.819103 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-5dedd956-5128-4773-ad0a-f714cc62ea79 admin admin] The token cache is being invalidate because project 94a3ca0d7d854e26a5e26ee48de284c4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81676) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 10 12:13:42.819390 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-5dedd956-5128-4773-ad0a-f714cc62ea79 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:13:42.820292 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-5dedd956-5128-4773-ad0a-f714cc62ea79 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:13:42.823012 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1120/2263] 10.222.0.46 () {58 vars in 1233 bytes} [Mon Oct 10 12:13:42 2022] DELETE /identity/v3/projects/94a3ca0d7d854e26a5e26ee48de284c4 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:13:42.827816 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df841be4-7314-453f-8b01-a7e927db4d3a None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:13:42.828305 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df841be4-7314-453f-8b01-a7e927db4d3a None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:13:42.828575 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df841be4-7314-453f-8b01-a7e927db4d3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:13:42.853085 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-df841be4-7314-453f-8b01-a7e927db4d3a None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:13:42.854299 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-df841be4-7314-453f-8b01-a7e927db4d3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:13:42.872289 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:13:42.917061 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1144/2264] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:13:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:13:42.926694 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05099187-e1b8-4dfc-8763-8e3d0ee1929d None None] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:13:42.927233 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05099187-e1b8-4dfc-8763-8e3d0ee1929d None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:13:42.927744 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05099187-e1b8-4dfc-8763-8e3d0ee1929d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:13:42.961688 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.auth.core [None req-05099187-e1b8-4dfc-8763-8e3d0ee1929d None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:13:42.963881 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-05099187-e1b8-4dfc-8763-8e3d0ee1929d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:13:42.977957 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1121/2265] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:13:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:13:42.982166 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df841be4-7314-453f-8b01-a7e927db4d3a None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:13:42.992505 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df841be4-7314-453f-8b01-a7e927db4d3a None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:13:42.998357 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6464016-52b1-4475-8077-6f6fd0b3eb0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:13:42.999095 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6464016-52b1-4475-8077-6f6fd0b3eb0a admin admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:13:43.004882 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6464016-52b1-4475-8077-6f6fd0b3eb0a admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:13:43.004882 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6464016-52b1-4475-8077-6f6fd0b3eb0a admin admin] PATH_INFO: `/v3/domains` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:13:43.004882 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6464016-52b1-4475-8077-6f6fd0b3eb0a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 12:13:43.004882 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6464016-52b1-4475-8077-6f6fd0b3eb0a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:13:43.004882 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6464016-52b1-4475-8077-6f6fd0b3eb0a admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:13:43.014956 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1145/2266] 10.222.0.46 () {58 vars in 1153 bytes} [Mon Oct 10 12:13:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 12:13:43.020207 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05099187-e1b8-4dfc-8763-8e3d0ee1929d None None] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:13:43.030393 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05099187-e1b8-4dfc-8763-8e3d0ee1929d None None] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:13:43.035450 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da1767b-396b-4574-874f-b191329b01be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:13:43.036067 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da1767b-396b-4574-874f-b191329b01be admin admin] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:13:43.036351 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da1767b-396b-4574-874f-b191329b01be admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:13:43.036626 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da1767b-396b-4574-874f-b191329b01be admin admin] PATH_INFO: `/v3/projects` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:13:43.037345 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da1767b-396b-4574-874f-b191329b01be admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:13:43.039432 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da1767b-396b-4574-874f-b191329b01be admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:13:43.153452 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1122/2267] 10.222.0.46 () {60 vars in 1153 bytes} [Mon Oct 10 12:13:43 2022] POST /identity/v3/projects => generated 389 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 12:13:43.158025 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6464016-52b1-4475-8077-6f6fd0b3eb0a admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:13:43.168662 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6464016-52b1-4475-8077-6f6fd0b3eb0a admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:13:43.223680 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e2ede5-6299-4f37-a2dc-e8e4d963f9ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:13:43.224289 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e2ede5-6299-4f37-a2dc-e8e4d963f9ea admin admin] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:13:43.224566 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e2ede5-6299-4f37-a2dc-e8e4d963f9ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:13:43.224836 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e2ede5-6299-4f37-a2dc-e8e4d963f9ea admin admin] PATH_INFO: `/v3/users` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:13:43.225474 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e2ede5-6299-4f37-a2dc-e8e4d963f9ea admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:13:43.227920 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e2ede5-6299-4f37-a2dc-e8e4d963f9ea admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:13:43.361567 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1146/2268] 10.222.0.46 () {60 vars in 1144 bytes} [Mon Oct 10 12:13:43 2022] POST /identity/v3/users => generated 295 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 12:13:43.365761 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da1767b-396b-4574-874f-b191329b01be admin admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:13:43.376624 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da1767b-396b-4574-874f-b191329b01be admin admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:13:43.381322 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292dc79f-4960-4ee7-b727-70bc5ce62965 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:13:43.381930 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292dc79f-4960-4ee7-b727-70bc5ce62965 admin admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:13:43.382211 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292dc79f-4960-4ee7-b727-70bc5ce62965 admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:13:43.382481 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292dc79f-4960-4ee7-b727-70bc5ce62965 admin admin] PATH_INFO: `/v3/roles` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:13:43.383170 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292dc79f-4960-4ee7-b727-70bc5ce62965 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:13:43.384830 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292dc79f-4960-4ee7-b727-70bc5ce62965 admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:13:43.395318 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1123/2269] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 10 12:13:43 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 12:13:43.399988 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e2ede5-6299-4f37-a2dc-e8e4d963f9ea admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:13:43.410271 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e2ede5-6299-4f37-a2dc-e8e4d963f9ea admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:13:43.415795 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604a638b-7c33-4bdd-adf2-2e22355f7f04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:13:43.416397 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604a638b-7c33-4bdd-adf2-2e22355f7f04 admin admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:13:43.416674 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604a638b-7c33-4bdd-adf2-2e22355f7f04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:13:43.416946 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604a638b-7c33-4bdd-adf2-2e22355f7f04 admin admin] PATH_INFO: `/v3/roles` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:13:43.417592 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604a638b-7c33-4bdd-adf2-2e22355f7f04 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:13:43.419265 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604a638b-7c33-4bdd-adf2-2e22355f7f04 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:13:43.429910 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1147/2270] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 10 12:13:43 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 12:13:43.435030 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292dc79f-4960-4ee7-b727-70bc5ce62965 admin admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:13:43.445430 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292dc79f-4960-4ee7-b727-70bc5ce62965 admin admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:13:43.454952 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e2e72e-d7e2-4f65-b86a-c911576932a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:13:43.455637 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e2e72e-d7e2-4f65-b86a-c911576932a2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:13:43.455907 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e2e72e-d7e2-4f65-b86a-c911576932a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:13:43.456179 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e2e72e-d7e2-4f65-b86a-c911576932a2 admin admin] PATH_INFO: `/v3/projects/a7bbf361e48c4f8ebb39d085587c0811/users/1f1cc07e465944b3b2bdb4852a6f0da7/roles/b2cb2a6e818e47338ce071e6fedfc673` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:13:43.472326 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e2e72e-d7e2-4f65-b86a-c911576932a2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a7bbf361e48c4f8ebb39d085587c0811, user_id=1f1cc07e465944b3b2bdb4852a6f0da7, role_id=b2cb2a6e818e47338ce071e6fedfc673)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:13:43.474964 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e2e72e-d7e2-4f65-b86a-c911576932a2 admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:13:43.630036 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1124/2271] 10.222.0.46 () {60 vars in 1483 bytes} [Mon Oct 10 12:13:43 2022] PUT /identity/v3/projects/a7bbf361e48c4f8ebb39d085587c0811/users/1f1cc07e465944b3b2bdb4852a6f0da7/roles/b2cb2a6e818e47338ce071e6fedfc673 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 10 12:13:43.641615 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d62e21-1bf5-4554-a1ff-8302fc6e37e8 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:13:43.645512 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d62e21-1bf5-4554-a1ff-8302fc6e37e8 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:13:43.645512 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d62e21-1bf5-4554-a1ff-8302fc6e37e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:13:43.670887 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-17d62e21-1bf5-4554-a1ff-8302fc6e37e8 None None] MFA Rules not processed for user `1f1cc07e465944b3b2bdb4852a6f0da7`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:13:43.672019 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-17d62e21-1bf5-4554-a1ff-8302fc6e37e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:13:43.742064 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1148/2272] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:13:43 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:13:43.753671 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e2e72e-d7e2-4f65-b86a-c911576932a2 admin admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:13:43.755605 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-20e2e72e-d7e2-4f65-b86a-c911576932a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:13:43.766632 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:13:43.816307 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e2e72e-d7e2-4f65-b86a-c911576932a2 admin admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:13:43.821051 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b319fa-393c-4d14-9f10-1450f3a166f4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3ae093470484b93b23dcfae1657df9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad3772f7b6a47c28b7ba37a9316a3c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:13:43.821359 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b319fa-393c-4d14-9f10-1450f3a166f4 service cinder] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:13:43.821637 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b319fa-393c-4d14-9f10-1450f3a166f4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:13:43.821927 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b319fa-393c-4d14-9f10-1450f3a166f4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:13:43.832376 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b319fa-393c-4d14-9f10-1450f3a166f4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:13:43.833362 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b319fa-393c-4d14-9f10-1450f3a166f4 service cinder] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:13:43.845092 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1125/2273] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 10 12:13:43 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:13:53.737418 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=89,keystone:INSERT=3 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:13:54.212692 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=23 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:13:55.309735 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d62e21-1bf5-4554-a1ff-8302fc6e37e8 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:13:55.311450 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-17d62e21-1bf5-4554-a1ff-8302fc6e37e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:13:55.396197 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d62e21-1bf5-4554-a1ff-8302fc6e37e8 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:13:55.403985 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07fe015-b576-44d2-b3a9-88c14ac57722 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f130bbb69e546ce99b4e14de81d7e54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad3772f7b6a47c28b7ba37a9316a3c1', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:13:55.404378 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07fe015-b576-44d2-b3a9-88c14ac57722 service nova] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:13:55.404714 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07fe015-b576-44d2-b3a9-88c14ac57722 service nova] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:13:55.405470 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07fe015-b576-44d2-b3a9-88c14ac57722 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:13:55.407505 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-c07fe015-b576-44d2-b3a9-88c14ac57722 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:13:55.420599 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07fe015-b576-44d2-b3a9-88c14ac57722 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:13:55.426911 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07fe015-b576-44d2-b3a9-88c14ac57722 service nova] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:13:55.455721 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1149/2274] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 10 12:13:55 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:14:05.450583 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=34 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:14:14.069436 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b319fa-393c-4d14-9f10-1450f3a166f4 service cinder] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:14:14.070858 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-28b319fa-393c-4d14-9f10-1450f3a166f4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:14:14.138330 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b319fa-393c-4d14-9f10-1450f3a166f4 service cinder] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:14:14.144064 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a94aadc-7a37-4b88-9eee-19262329319b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f130bbb69e546ce99b4e14de81d7e54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad3772f7b6a47c28b7ba37a9316a3c1', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:14:14.144676 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a94aadc-7a37-4b88-9eee-19262329319b service nova] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:14:14.144954 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a94aadc-7a37-4b88-9eee-19262329319b service nova] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:14:14.145225 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a94aadc-7a37-4b88-9eee-19262329319b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:14:14.147000 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-1a94aadc-7a37-4b88-9eee-19262329319b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:14:14.210078 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a94aadc-7a37-4b88-9eee-19262329319b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:14:14.211669 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a94aadc-7a37-4b88-9eee-19262329319b service nova] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:14:14.221728 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1126/2275] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 10 12:14:14 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:14:20.882602 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7939578-9f57-4f50-96b6-f92b8968e30e None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:14:20.889404 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7939578-9f57-4f50-96b6-f92b8968e30e None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:14:20.889404 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7939578-9f57-4f50-96b6-f92b8968e30e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:14:20.914127 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-f7939578-9f57-4f50-96b6-f92b8968e30e None None] MFA Rules not processed for user `d6430593eecf45d8b891c3b55de4a944`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:14:20.915518 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-f7939578-9f57-4f50-96b6-f92b8968e30e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:14:20.972434 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1150/2276] 10.222.0.46 () {58 vars in 967 bytes} [Mon Oct 10 12:14:20 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:14:21.014621 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a94aadc-7a37-4b88-9eee-19262329319b service nova] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:14:21.016076 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-1a94aadc-7a37-4b88-9eee-19262329319b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:14:21.084039 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a94aadc-7a37-4b88-9eee-19262329319b service nova] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:14:21.089519 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9af24f-99e7-4dbe-8d41-01acc2ed8262 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15ef3d53ea484237adb0b3d64fdc5480', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad3772f7b6a47c28b7ba37a9316a3c1', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:14:21.090155 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9af24f-99e7-4dbe-8d41-01acc2ed8262 service swift] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:14:21.090438 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9af24f-99e7-4dbe-8d41-01acc2ed8262 service swift] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:14:21.090746 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9af24f-99e7-4dbe-8d41-01acc2ed8262 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:14:21.099445 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9af24f-99e7-4dbe-8d41-01acc2ed8262 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:14:21.101024 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9af24f-99e7-4dbe-8d41-01acc2ed8262 service swift] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:14:21.109610 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1127/2277] 10.222.0.46 () {60 vars in 1384 bytes} [Mon Oct 10 12:14:21 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 10 12:14:25.474076 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7939578-9f57-4f50-96b6-f92b8968e30e None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:14:25.556536 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7939578-9f57-4f50-96b6-f92b8968e30e None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:14:25.561212 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5273345e-c83f-407c-84e8-fb38bcc2cf2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:14:25.561823 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5273345e-c83f-407c-84e8-fb38bcc2cf2e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:14:25.562101 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5273345e-c83f-407c-84e8-fb38bcc2cf2e admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:14:25.562381 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5273345e-c83f-407c-84e8-fb38bcc2cf2e admin admin] PATH_INFO: `/v3/users/1f1cc07e465944b3b2bdb4852a6f0da7` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:14:25.563544 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5273345e-c83f-407c-84e8-fb38bcc2cf2e admin admin] RBAC: Authorizing `identity:delete_user(user_id=1f1cc07e465944b3b2bdb4852a6f0da7)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:14:25.565481 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5273345e-c83f-407c-84e8-fb38bcc2cf2e admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:14:25.747177 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-5273345e-c83f-407c-84e8-fb38bcc2cf2e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1f1cc07e465944b3b2bdb4852a6f0da7'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:14:25.754769 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-5273345e-c83f-407c-84e8-fb38bcc2cf2e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1f1cc07e465944b3b2bdb4852a6f0da7'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:14:25.755517 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-5273345e-c83f-407c-84e8-fb38bcc2cf2e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1f1cc07e465944b3b2bdb4852a6f0da7'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:14:25.846800 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-5273345e-c83f-407c-84e8-fb38bcc2cf2e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1f1cc07e465944b3b2bdb4852a6f0da7'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:14:25.859603 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-5273345e-c83f-407c-84e8-fb38bcc2cf2e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1f1cc07e465944b3b2bdb4852a6f0da7'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:14:25.862515 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:14:25.866292 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1151/2278] 10.222.0.46 () {58 vars in 1224 bytes} [Mon Oct 10 12:14:25 2022] DELETE /identity/v3/users/1f1cc07e465944b3b2bdb4852a6f0da7 => generated 0 bytes in 393 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:14:25.870908 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abee189e-7180-435e-9317-1d91c0b2aa02 None None] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:14:25.871396 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abee189e-7180-435e-9317-1d91c0b2aa02 None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:14:25.871526 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abee189e-7180-435e-9317-1d91c0b2aa02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:14:25.912120 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.auth.core [None req-abee189e-7180-435e-9317-1d91c0b2aa02 None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:14:25.913317 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-abee189e-7180-435e-9317-1d91c0b2aa02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:14:25.929639 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:14:25.973251 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1128/2279] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:14:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:14:25.983007 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5273345e-c83f-407c-84e8-fb38bcc2cf2e admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:14:25.984494 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-5273345e-c83f-407c-84e8-fb38bcc2cf2e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:14:26.064359 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5273345e-c83f-407c-84e8-fb38bcc2cf2e admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:14:26.068026 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280bdd55-cb46-482a-862d-43396f335ba2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58b1c7c85302421cba823ace4201861f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad3772f7b6a47c28b7ba37a9316a3c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:14:26.068666 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280bdd55-cb46-482a-862d-43396f335ba2 service neutron] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:14:26.068952 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280bdd55-cb46-482a-862d-43396f335ba2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:14:26.069226 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280bdd55-cb46-482a-862d-43396f335ba2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:14:26.079162 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280bdd55-cb46-482a-862d-43396f335ba2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:14:26.080581 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280bdd55-cb46-482a-862d-43396f335ba2 service neutron] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:14:26.090145 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1152/2280] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 10 12:14:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:14:26.534267 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abee189e-7180-435e-9317-1d91c0b2aa02 None None] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:14:26.535793 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-abee189e-7180-435e-9317-1d91c0b2aa02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:14:26.551072 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abee189e-7180-435e-9317-1d91c0b2aa02 None None] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:14:26.556103 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4508cd9d-aacd-40b7-ba18-a2a90d8a7988 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:14:26.556747 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4508cd9d-aacd-40b7-ba18-a2a90d8a7988 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:14:26.557030 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4508cd9d-aacd-40b7-ba18-a2a90d8a7988 admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:14:26.557310 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4508cd9d-aacd-40b7-ba18-a2a90d8a7988 admin admin] PATH_INFO: `/v3/projects/a7bbf361e48c4f8ebb39d085587c0811` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:14:26.558448 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4508cd9d-aacd-40b7-ba18-a2a90d8a7988 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a7bbf361e48c4f8ebb39d085587c0811)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:14:26.560981 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4508cd9d-aacd-40b7-ba18-a2a90d8a7988 admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:14:26.674079 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-4508cd9d-aacd-40b7-ba18-a2a90d8a7988 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a7bbf361e48c4f8ebb39d085587c0811', 'request_id': 'req-4508cd9d-aacd-40b7-ba18-a2a90d8a7988'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:14:26.683239 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-4508cd9d-aacd-40b7-ba18-a2a90d8a7988 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a7bbf361e48c4f8ebb39d085587c0811', 'request_id': 'req-4508cd9d-aacd-40b7-ba18-a2a90d8a7988'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:14:26.692586 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-4508cd9d-aacd-40b7-ba18-a2a90d8a7988 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a7bbf361e48c4f8ebb39d085587c0811', 'request_id': 'req-4508cd9d-aacd-40b7-ba18-a2a90d8a7988'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:14:26.823169 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-4508cd9d-aacd-40b7-ba18-a2a90d8a7988 admin admin] The token cache is being invalidate because project a7bbf361e48c4f8ebb39d085587c0811 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81676) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 10 12:14:26.823462 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-4508cd9d-aacd-40b7-ba18-a2a90d8a7988 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:14:26.824118 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-4508cd9d-aacd-40b7-ba18-a2a90d8a7988 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:14:26.825678 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1129/2281] 10.222.0.46 () {58 vars in 1233 bytes} [Mon Oct 10 12:14:26 2022] DELETE /identity/v3/projects/a7bbf361e48c4f8ebb39d085587c0811 => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:14:31.112862 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280bdd55-cb46-482a-862d-43396f335ba2 service neutron] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:14:31.117285 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-280bdd55-cb46-482a-862d-43396f335ba2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:14:31.191758 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280bdd55-cb46-482a-862d-43396f335ba2 service neutron] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:14:31.257838 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224b21e6-b821-4cdf-8f2a-ee081ea16eb9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f130bbb69e546ce99b4e14de81d7e54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad3772f7b6a47c28b7ba37a9316a3c1', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:14:31.258450 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224b21e6-b821-4cdf-8f2a-ee081ea16eb9 service nova] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:14:31.258762 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224b21e6-b821-4cdf-8f2a-ee081ea16eb9 service nova] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:14:31.259064 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224b21e6-b821-4cdf-8f2a-ee081ea16eb9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:14:31.260800 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-224b21e6-b821-4cdf-8f2a-ee081ea16eb9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:14:31.272505 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224b21e6-b821-4cdf-8f2a-ee081ea16eb9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:14:31.274124 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224b21e6-b821-4cdf-8f2a-ee081ea16eb9 service nova] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:14:31.285427 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1153/2282] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 10 12:14:31 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:14:36.819683 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=44,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:14:41.281394 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=98 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:14:47.555337 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4508cd9d-aacd-40b7-ba18-a2a90d8a7988 admin admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:14:47.556833 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-4508cd9d-aacd-40b7-ba18-a2a90d8a7988 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:14:47.649391 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4508cd9d-aacd-40b7-ba18-a2a90d8a7988 admin admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:14:47.653213 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc016e4b-d53d-4a64-af7c-3c1711f5202b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3ae093470484b93b23dcfae1657df9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad3772f7b6a47c28b7ba37a9316a3c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:14:47.653913 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc016e4b-d53d-4a64-af7c-3c1711f5202b service cinder] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:14:47.654340 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc016e4b-d53d-4a64-af7c-3c1711f5202b service cinder] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:14:47.654662 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc016e4b-d53d-4a64-af7c-3c1711f5202b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:14:47.660031 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-bc016e4b-d53d-4a64-af7c-3c1711f5202b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:14:47.725883 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc016e4b-d53d-4a64-af7c-3c1711f5202b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:14:47.727392 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc016e4b-d53d-4a64-af7c-3c1711f5202b service cinder] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:14:47.738628 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1130/2283] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 10 12:14:47 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:14:57.732766 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=57 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:15:07.236008 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224b21e6-b821-4cdf-8f2a-ee081ea16eb9 service nova] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:15:07.237465 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-224b21e6-b821-4cdf-8f2a-ee081ea16eb9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:15:07.294363 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224b21e6-b821-4cdf-8f2a-ee081ea16eb9 service nova] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:15:07.298202 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e04ed3-883a-447c-a48a-34d81853c19d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58b1c7c85302421cba823ace4201861f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad3772f7b6a47c28b7ba37a9316a3c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:15:07.298843 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e04ed3-883a-447c-a48a-34d81853c19d service neutron] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:15:07.299127 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e04ed3-883a-447c-a48a-34d81853c19d service neutron] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:15:07.299400 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e04ed3-883a-447c-a48a-34d81853c19d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:15:07.301155 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-76e04ed3-883a-447c-a48a-34d81853c19d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:15:07.365923 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e04ed3-883a-447c-a48a-34d81853c19d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:15:07.367385 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e04ed3-883a-447c-a48a-34d81853c19d service neutron] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:15:07.377252 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1154/2284] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 10 12:15:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:15:07.886042 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc016e4b-d53d-4a64-af7c-3c1711f5202b service cinder] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:15:07.887565 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-bc016e4b-d53d-4a64-af7c-3c1711f5202b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:15:07.958440 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc016e4b-d53d-4a64-af7c-3c1711f5202b service cinder] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:15:07.964053 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00209309-b801-4d6e-991c-d4da9f7779d0 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e8f9bb1776d47169ef7febb57c954cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad3772f7b6a47c28b7ba37a9316a3c1', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:15:07.964661 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00209309-b801-4d6e-991c-d4da9f7779d0 service placement] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:15:07.964996 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00209309-b801-4d6e-991c-d4da9f7779d0 service placement] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:15:07.965277 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00209309-b801-4d6e-991c-d4da9f7779d0 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:15:07.967082 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-00209309-b801-4d6e-991c-d4da9f7779d0 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:15:07.979261 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00209309-b801-4d6e-991c-d4da9f7779d0 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:15:07.980833 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00209309-b801-4d6e-991c-d4da9f7779d0 service placement] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:15:07.991215 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1131/2285] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 10 12:15:07 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:15:08.000788 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e04ed3-883a-447c-a48a-34d81853c19d service neutron] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:15:08.005228 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-76e04ed3-883a-447c-a48a-34d81853c19d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:15:08.018792 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e04ed3-883a-447c-a48a-34d81853c19d service neutron] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:15:08.024507 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07581fd2-075d-4302-bb63-770f00e26c4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:15:08.025155 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07581fd2-075d-4302-bb63-770f00e26c4e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:15:08.025433 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07581fd2-075d-4302-bb63-770f00e26c4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:15:08.025709 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07581fd2-075d-4302-bb63-770f00e26c4e admin admin] PATH_INFO: `/v3/users/13ac3580c0164d20994d68744d9153b8` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:15:08.027242 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07581fd2-075d-4302-bb63-770f00e26c4e admin admin] RBAC: Authorizing `identity:delete_user(user_id=13ac3580c0164d20994d68744d9153b8)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:15:08.029250 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07581fd2-075d-4302-bb63-770f00e26c4e admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:15:08.227134 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-07581fd2-075d-4302-bb63-770f00e26c4e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '13ac3580c0164d20994d68744d9153b8'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:15:08.234210 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-07581fd2-075d-4302-bb63-770f00e26c4e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '13ac3580c0164d20994d68744d9153b8'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:15:08.234933 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-07581fd2-075d-4302-bb63-770f00e26c4e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '13ac3580c0164d20994d68744d9153b8'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:15:08.320426 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-07581fd2-075d-4302-bb63-770f00e26c4e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '13ac3580c0164d20994d68744d9153b8'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:15:08.333326 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-07581fd2-075d-4302-bb63-770f00e26c4e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '13ac3580c0164d20994d68744d9153b8'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:15:08.339949 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1155/2286] 10.222.0.46 () {58 vars in 1224 bytes} [Mon Oct 10 12:15:07 2022] DELETE /identity/v3/users/13ac3580c0164d20994d68744d9153b8 => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:15:08.581508 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00209309-b801-4d6e-991c-d4da9f7779d0 service placement] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:15:08.593801 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00209309-b801-4d6e-991c-d4da9f7779d0 service placement] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:15:08.654888 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be46cd1-bc21-4069-aad6-15c562232e0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:15:08.655526 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be46cd1-bc21-4069-aad6-15c562232e0b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:15:08.655803 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be46cd1-bc21-4069-aad6-15c562232e0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:15:08.656084 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be46cd1-bc21-4069-aad6-15c562232e0b admin admin] PATH_INFO: `/v3/projects/42363f6f6a6040e5b81070350a2b7855` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:15:08.657219 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be46cd1-bc21-4069-aad6-15c562232e0b admin admin] RBAC: Authorizing `identity:delete_project(project_id=42363f6f6a6040e5b81070350a2b7855)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:15:08.659156 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be46cd1-bc21-4069-aad6-15c562232e0b admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:15:08.762891 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-4be46cd1-bc21-4069-aad6-15c562232e0b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '42363f6f6a6040e5b81070350a2b7855', 'request_id': 'req-4be46cd1-bc21-4069-aad6-15c562232e0b'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:15:08.769206 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-4be46cd1-bc21-4069-aad6-15c562232e0b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '42363f6f6a6040e5b81070350a2b7855', 'request_id': 'req-4be46cd1-bc21-4069-aad6-15c562232e0b'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:15:08.776273 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-4be46cd1-bc21-4069-aad6-15c562232e0b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '42363f6f6a6040e5b81070350a2b7855', 'request_id': 'req-4be46cd1-bc21-4069-aad6-15c562232e0b'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:15:08.873786 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-4be46cd1-bc21-4069-aad6-15c562232e0b admin admin] The token cache is being invalidate because project 42363f6f6a6040e5b81070350a2b7855 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81676) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 10 12:15:08.874077 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-4be46cd1-bc21-4069-aad6-15c562232e0b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:15:08.874762 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-4be46cd1-bc21-4069-aad6-15c562232e0b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:15:08.876302 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1132/2287] 10.222.0.46 () {58 vars in 1233 bytes} [Mon Oct 10 12:15:08 2022] DELETE /identity/v3/projects/42363f6f6a6040e5b81070350a2b7855 => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:15:15.963484 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8943f374-7047-4e9e-972a-e390b42d8575 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:15:15.964013 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8943f374-7047-4e9e-972a-e390b42d8575 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:15:15.964013 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8943f374-7047-4e9e-972a-e390b42d8575 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:15:15.983749 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-8943f374-7047-4e9e-972a-e390b42d8575 None None] MFA Rules not processed for user `45c87aa85b7d45fe8acf6f36a3d2a639`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:15:15.984980 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-8943f374-7047-4e9e-972a-e390b42d8575 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:15:16.002576 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:15:16.042373 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1156/2288] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:15:15 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:15:18.870395 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=83,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:15:26.136592 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=15 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:15:56.605239 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be46cd1-bc21-4069-aad6-15c562232e0b admin admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:15:56.607555 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-4be46cd1-bc21-4069-aad6-15c562232e0b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:15:56.738532 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be46cd1-bc21-4069-aad6-15c562232e0b admin admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:15:56.743951 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442787ed-9aa6-4c22-a208-157b85f71285 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e8f9bb1776d47169ef7febb57c954cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad3772f7b6a47c28b7ba37a9316a3c1', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:15:56.744575 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442787ed-9aa6-4c22-a208-157b85f71285 service placement] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:15:56.744892 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442787ed-9aa6-4c22-a208-157b85f71285 service placement] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:15:56.745165 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442787ed-9aa6-4c22-a208-157b85f71285 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:15:56.746969 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-442787ed-9aa6-4c22-a208-157b85f71285 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:15:56.760675 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442787ed-9aa6-4c22-a208-157b85f71285 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:15:56.762341 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442787ed-9aa6-4c22-a208-157b85f71285 service placement] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:15:56.773576 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1133/2289] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 10 12:15:56 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:16:03.230405 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97a21e5-3d6d-4976-80b2-5e6a0f15dad3 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:16:03.231054 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97a21e5-3d6d-4976-80b2-5e6a0f15dad3 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:16:03.231237 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97a21e5-3d6d-4976-80b2-5e6a0f15dad3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:16:03.250763 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-b97a21e5-3d6d-4976-80b2-5e6a0f15dad3 None None] MFA Rules not processed for user `fef422264d0c4a43a0aa1b9eff2251bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:16:03.251972 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-b97a21e5-3d6d-4976-80b2-5e6a0f15dad3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:16:03.339785 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1157/2290] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:16:03 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:16:03.672177 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442787ed-9aa6-4c22-a208-157b85f71285 service placement] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:16:03.673820 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-442787ed-9aa6-4c22-a208-157b85f71285 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:16:03.747498 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442787ed-9aa6-4c22-a208-157b85f71285 service placement] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:16:03.751313 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542750f7-5f99-445d-9789-d0ef165b1d55 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58b1c7c85302421cba823ace4201861f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad3772f7b6a47c28b7ba37a9316a3c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:16:03.751926 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542750f7-5f99-445d-9789-d0ef165b1d55 service neutron] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:16:03.752212 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542750f7-5f99-445d-9789-d0ef165b1d55 service neutron] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:16:03.752490 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542750f7-5f99-445d-9789-d0ef165b1d55 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:16:03.754265 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-542750f7-5f99-445d-9789-d0ef165b1d55 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:16:03.765098 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542750f7-5f99-445d-9789-d0ef165b1d55 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:16:03.766562 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542750f7-5f99-445d-9789-d0ef165b1d55 service neutron] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:16:03.776182 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1134/2291] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 10 12:16:03 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:16:07.730035 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be724e1c-f120-4511-b12c-cd781eed65dc None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:16:07.730595 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be724e1c-f120-4511-b12c-cd781eed65dc None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:16:07.730595 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be724e1c-f120-4511-b12c-cd781eed65dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:16:07.750736 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-be724e1c-f120-4511-b12c-cd781eed65dc None None] MFA Rules not processed for user `a7a87488f78444c3ad820527e5fbc099`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:16:07.751952 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-be724e1c-f120-4511-b12c-cd781eed65dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:16:07.829086 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1158/2292] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:16:07 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:16:13.771606 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=67 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:16:17.805185 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=50 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:16:23.254501 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542750f7-5f99-445d-9789-d0ef165b1d55 service neutron] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:16:23.256075 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-542750f7-5f99-445d-9789-d0ef165b1d55 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:16:23.366330 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542750f7-5f99-445d-9789-d0ef165b1d55 service neutron] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:16:23.373112 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4121c2db-b66f-4310-9115-210e77c64696 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f130bbb69e546ce99b4e14de81d7e54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad3772f7b6a47c28b7ba37a9316a3c1', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:16:23.374009 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4121c2db-b66f-4310-9115-210e77c64696 service nova] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:16:23.374329 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4121c2db-b66f-4310-9115-210e77c64696 service nova] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:16:23.374630 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4121c2db-b66f-4310-9115-210e77c64696 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:16:23.376491 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-4121c2db-b66f-4310-9115-210e77c64696 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:16:23.391029 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4121c2db-b66f-4310-9115-210e77c64696 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:16:23.392868 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4121c2db-b66f-4310-9115-210e77c64696 service nova] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:16:23.404827 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1135/2293] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 10 12:16:23 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:16:33.400040 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=46 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:16:43.561971 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be724e1c-f120-4511-b12c-cd781eed65dc None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:16:43.582251 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be724e1c-f120-4511-b12c-cd781eed65dc None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:16:43.591221 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25175406-13cc-4218-b042-8c2d01e2c6e5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f130bbb69e546ce99b4e14de81d7e54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad3772f7b6a47c28b7ba37a9316a3c1', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:16:43.592187 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25175406-13cc-4218-b042-8c2d01e2c6e5 service nova] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:16:43.592528 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25175406-13cc-4218-b042-8c2d01e2c6e5 service nova] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:16:43.593066 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25175406-13cc-4218-b042-8c2d01e2c6e5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:16:43.594977 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-25175406-13cc-4218-b042-8c2d01e2c6e5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:16:43.656803 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25175406-13cc-4218-b042-8c2d01e2c6e5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:16:43.658382 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25175406-13cc-4218-b042-8c2d01e2c6e5 service nova] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:16:43.669258 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1159/2294] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 10 12:16:43 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 10 12:16:49.909650 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ecd164-23c5-4394-a42f-b350a8cebde5 None None] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:16:49.910447 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ecd164-23c5-4394-a42f-b350a8cebde5 None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:16:49.910447 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ecd164-23c5-4394-a42f-b350a8cebde5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:16:49.930943 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.auth.core [None req-86ecd164-23c5-4394-a42f-b350a8cebde5 None None] MFA Rules not processed for user `21c537b0fefe4fb1b51483ac0e3cf2d1`. Rule list: `[]` (Enabled: `True`). {{(pid=81676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:16:49.932403 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-86ecd164-23c5-4394-a42f-b350a8cebde5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:16:49.939440 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1136/2295] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:16:49 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:16:53.424596 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25175406-13cc-4218-b042-8c2d01e2c6e5 service nova] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:16:53.426021 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-25175406-13cc-4218-b042-8c2d01e2c6e5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:16:53.441504 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25175406-13cc-4218-b042-8c2d01e2c6e5 service nova] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:16:53.446702 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aefeb3a-4188-4367-8fc4-8315de8e249c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58b1c7c85302421cba823ace4201861f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad3772f7b6a47c28b7ba37a9316a3c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:16:53.448038 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aefeb3a-4188-4367-8fc4-8315de8e249c service neutron] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:16:53.448329 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aefeb3a-4188-4367-8fc4-8315de8e249c service neutron] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:16:53.448606 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aefeb3a-4188-4367-8fc4-8315de8e249c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:16:53.450904 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-8aefeb3a-4188-4367-8fc4-8315de8e249c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:16:53.594890 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aefeb3a-4188-4367-8fc4-8315de8e249c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:16:53.596657 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aefeb3a-4188-4367-8fc4-8315de8e249c service neutron] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:16:53.609614 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1160/2296] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 10 12:16:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 10 12:16:59.924315 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=3 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:17:00.006608 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ecd164-23c5-4394-a42f-b350a8cebde5 None None] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:17:00.008307 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-86ecd164-23c5-4394-a42f-b350a8cebde5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:17:00.029055 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ecd164-23c5-4394-a42f-b350a8cebde5 None None] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:17:00.034256 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42276bb-8429-476a-955a-f244ecd1cd8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:17:00.034980 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42276bb-8429-476a-955a-f244ecd1cd8d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:17:00.035261 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42276bb-8429-476a-955a-f244ecd1cd8d admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:17:00.035536 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42276bb-8429-476a-955a-f244ecd1cd8d admin admin] PATH_INFO: `/v3/users/45c87aa85b7d45fe8acf6f36a3d2a639` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:17:00.036800 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42276bb-8429-476a-955a-f244ecd1cd8d admin admin] RBAC: Authorizing `identity:delete_user(user_id=45c87aa85b7d45fe8acf6f36a3d2a639)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:17:00.038860 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42276bb-8429-476a-955a-f244ecd1cd8d admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:17:00.159983 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f1458f-9ee4-4256-817d-275124c6b404 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:17:00.160788 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f1458f-9ee4-4256-817d-275124c6b404 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:17:00.160788 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f1458f-9ee4-4256-817d-275124c6b404 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:17:00.181302 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-05f1458f-9ee4-4256-817d-275124c6b404 None None] MFA Rules not processed for user `b334b2401e14419284cce2ad61442d4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:17:00.182556 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-05f1458f-9ee4-4256-817d-275124c6b404 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:17:00.219620 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-d42276bb-8429-476a-955a-f244ecd1cd8d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '45c87aa85b7d45fe8acf6f36a3d2a639'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:17:00.228633 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-d42276bb-8429-476a-955a-f244ecd1cd8d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '45c87aa85b7d45fe8acf6f36a3d2a639'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:17:00.229387 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-d42276bb-8429-476a-955a-f244ecd1cd8d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '45c87aa85b7d45fe8acf6f36a3d2a639'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:17:00.247890 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1161/2297] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:17:00 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 10 12:17:00.301614 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-d42276bb-8429-476a-955a-f244ecd1cd8d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '45c87aa85b7d45fe8acf6f36a3d2a639'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:17:00.317013 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-d42276bb-8429-476a-955a-f244ecd1cd8d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '45c87aa85b7d45fe8acf6f36a3d2a639'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:17:00.324365 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1137/2298] 10.222.0.46 () {58 vars in 1224 bytes} [Mon Oct 10 12:17:00 2022] DELETE /identity/v3/users/45c87aa85b7d45fe8acf6f36a3d2a639 => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:17:00.646891 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f1458f-9ee4-4256-817d-275124c6b404 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:17:00.649778 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-05f1458f-9ee4-4256-817d-275124c6b404 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:17:00.669792 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f1458f-9ee4-4256-817d-275124c6b404 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:17:00.725686 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:17:00.759007 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6427c63-ca6d-43ce-b3e4-6a8ddbf41482 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:17:00.760753 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6427c63-ca6d-43ce-b3e4-6a8ddbf41482 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:17:00.760753 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6427c63-ca6d-43ce-b3e4-6a8ddbf41482 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:17:00.760753 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6427c63-ca6d-43ce-b3e4-6a8ddbf41482 admin admin] PATH_INFO: `/v3/projects/782c5fd84235487caac5f9a7e18563cb` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:17:00.762328 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6427c63-ca6d-43ce-b3e4-6a8ddbf41482 admin admin] RBAC: Authorizing `identity:delete_project(project_id=782c5fd84235487caac5f9a7e18563cb)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:17:00.763888 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6427c63-ca6d-43ce-b3e4-6a8ddbf41482 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:17:00.875812 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-f6427c63-ca6d-43ce-b3e4-6a8ddbf41482 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '782c5fd84235487caac5f9a7e18563cb', 'request_id': 'req-f6427c63-ca6d-43ce-b3e4-6a8ddbf41482'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:17:00.883510 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-f6427c63-ca6d-43ce-b3e4-6a8ddbf41482 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '782c5fd84235487caac5f9a7e18563cb', 'request_id': 'req-f6427c63-ca6d-43ce-b3e4-6a8ddbf41482'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:17:00.891562 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-f6427c63-ca6d-43ce-b3e4-6a8ddbf41482 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '782c5fd84235487caac5f9a7e18563cb', 'request_id': 'req-f6427c63-ca6d-43ce-b3e4-6a8ddbf41482'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:17:00.971207 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-f6427c63-ca6d-43ce-b3e4-6a8ddbf41482 admin admin] The token cache is being invalidate because project 782c5fd84235487caac5f9a7e18563cb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81677) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 10 12:17:00.971624 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-f6427c63-ca6d-43ce-b3e4-6a8ddbf41482 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:17:00.972411 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-f6427c63-ca6d-43ce-b3e4-6a8ddbf41482 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:17:00.974176 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1162/2299] 10.222.0.46 () {58 vars in 1233 bytes} [Mon Oct 10 12:17:00 2022] DELETE /identity/v3/projects/782c5fd84235487caac5f9a7e18563cb => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:17:00.979993 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df120ba-3f88-45ee-a46c-c06e00df5e94 None None] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:17:00.981278 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df120ba-3f88-45ee-a46c-c06e00df5e94 None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:17:00.981658 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df120ba-3f88-45ee-a46c-c06e00df5e94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:17:01.039578 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.auth.core [None req-7df120ba-3f88-45ee-a46c-c06e00df5e94 None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:17:01.040802 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-7df120ba-3f88-45ee-a46c-c06e00df5e94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:17:01.105536 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1138/2300] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:17:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:17:01.113899 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3e2121-8cff-463c-a980-b3909d55a6d9 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:17:01.114201 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3e2121-8cff-463c-a980-b3909d55a6d9 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:17:01.114483 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3e2121-8cff-463c-a980-b3909d55a6d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:17:01.138482 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-5f3e2121-8cff-463c-a980-b3909d55a6d9 None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:17:01.138482 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-5f3e2121-8cff-463c-a980-b3909d55a6d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:17:01.152355 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1163/2301] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:17:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:17:01.158108 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df120ba-3f88-45ee-a46c-c06e00df5e94 None None] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:17:01.175903 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df120ba-3f88-45ee-a46c-c06e00df5e94 None None] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:17:01.182320 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8308db8-17fb-49e0-8643-a2e5dbdf3652 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:17:01.184278 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8308db8-17fb-49e0-8643-a2e5dbdf3652 admin admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:17:01.184760 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8308db8-17fb-49e0-8643-a2e5dbdf3652 admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:17:01.185213 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8308db8-17fb-49e0-8643-a2e5dbdf3652 admin admin] PATH_INFO: `/v3/domains` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:17:01.186365 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8308db8-17fb-49e0-8643-a2e5dbdf3652 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 12:17:01.186910 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8308db8-17fb-49e0-8643-a2e5dbdf3652 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:17:01.195143 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8308db8-17fb-49e0-8643-a2e5dbdf3652 admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:17:01.210197 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1139/2302] 10.222.0.46 () {58 vars in 1153 bytes} [Mon Oct 10 12:17:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 12:17:01.218308 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3e2121-8cff-463c-a980-b3909d55a6d9 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:17:01.234392 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3e2121-8cff-463c-a980-b3909d55a6d9 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:17:01.240603 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83ab18b-674f-4be9-abc0-00802d46e6b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:17:01.241297 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83ab18b-674f-4be9-abc0-00802d46e6b9 admin admin] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:17:01.241491 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83ab18b-674f-4be9-abc0-00802d46e6b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:17:01.241769 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83ab18b-674f-4be9-abc0-00802d46e6b9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:17:01.242810 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83ab18b-674f-4be9-abc0-00802d46e6b9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:17:01.244834 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83ab18b-674f-4be9-abc0-00802d46e6b9 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:17:01.367461 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1164/2303] 10.222.0.46 () {60 vars in 1153 bytes} [Mon Oct 10 12:17:01 2022] POST /identity/v3/projects => generated 371 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 12:17:01.373015 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8308db8-17fb-49e0-8643-a2e5dbdf3652 admin admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:17:01.388029 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8308db8-17fb-49e0-8643-a2e5dbdf3652 admin admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:17:01.429514 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:17:01.457556 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18e2497-ddae-4e5d-bca5-e1981e51aa4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:17:01.465827 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18e2497-ddae-4e5d-bca5-e1981e51aa4b admin admin] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:17:01.465827 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18e2497-ddae-4e5d-bca5-e1981e51aa4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:17:01.465827 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18e2497-ddae-4e5d-bca5-e1981e51aa4b admin admin] PATH_INFO: `/v3/users` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:17:01.465827 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18e2497-ddae-4e5d-bca5-e1981e51aa4b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:17:01.465827 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18e2497-ddae-4e5d-bca5-e1981e51aa4b admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:17:01.541808 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1140/2304] 10.222.0.46 () {60 vars in 1144 bytes} [Mon Oct 10 12:17:01 2022] POST /identity/v3/users => generated 286 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 12:17:01.546832 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83ab18b-674f-4be9-abc0-00802d46e6b9 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:17:01.558197 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83ab18b-674f-4be9-abc0-00802d46e6b9 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:17:01.563362 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4dc1a11-ad01-49fd-a3d6-9677d797288b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:17:01.563979 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dc1a11-ad01-49fd-a3d6-9677d797288b admin admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:17:01.564264 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dc1a11-ad01-49fd-a3d6-9677d797288b admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:17:01.564568 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dc1a11-ad01-49fd-a3d6-9677d797288b admin admin] PATH_INFO: `/v3/roles` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:17:01.565221 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4dc1a11-ad01-49fd-a3d6-9677d797288b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:17:01.566874 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4dc1a11-ad01-49fd-a3d6-9677d797288b admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:17:01.577770 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1165/2305] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 10 12:17:01 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 12:17:01.582323 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18e2497-ddae-4e5d-bca5-e1981e51aa4b admin admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:17:01.593418 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18e2497-ddae-4e5d-bca5-e1981e51aa4b admin admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:17:01.601415 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6a386e-5694-4311-973f-36893369ce31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:17:01.601415 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6a386e-5694-4311-973f-36893369ce31 admin admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:17:01.601415 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6a386e-5694-4311-973f-36893369ce31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:17:01.601415 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6a386e-5694-4311-973f-36893369ce31 admin admin] PATH_INFO: `/v3/roles` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:17:01.603662 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6a386e-5694-4311-973f-36893369ce31 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:17:01.603662 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6a386e-5694-4311-973f-36893369ce31 admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:17:01.616604 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1141/2306] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 10 12:17:01 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 12:17:01.621675 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4dc1a11-ad01-49fd-a3d6-9677d797288b admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:17:01.633124 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4dc1a11-ad01-49fd-a3d6-9677d797288b admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:17:01.639044 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94cc877-00f2-4912-af02-4e41e36415f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:17:01.639820 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94cc877-00f2-4912-af02-4e41e36415f4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:17:01.640225 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94cc877-00f2-4912-af02-4e41e36415f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:17:01.640689 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94cc877-00f2-4912-af02-4e41e36415f4 admin admin] PATH_INFO: `/v3/projects/3ffdb088d2f7493996841f5ef0d4b612/users/65da864252a347a8a0c65f15597d2e8b/roles/b2cb2a6e818e47338ce071e6fedfc673` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:17:01.663425 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94cc877-00f2-4912-af02-4e41e36415f4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3ffdb088d2f7493996841f5ef0d4b612, user_id=65da864252a347a8a0c65f15597d2e8b, role_id=b2cb2a6e818e47338ce071e6fedfc673)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:17:01.666643 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94cc877-00f2-4912-af02-4e41e36415f4 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:17:01.815649 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1166/2307] 10.222.0.46 () {60 vars in 1483 bytes} [Mon Oct 10 12:17:01 2022] PUT /identity/v3/projects/3ffdb088d2f7493996841f5ef0d4b612/users/65da864252a347a8a0c65f15597d2e8b/roles/b2cb2a6e818e47338ce071e6fedfc673 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:17:01.829209 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ec74b8-6aab-4e2e-ad5c-625d25c26ccc None None] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:17:01.829875 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ec74b8-6aab-4e2e-ad5c-625d25c26ccc None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:17:01.829875 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ec74b8-6aab-4e2e-ad5c-625d25c26ccc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:17:01.861112 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.auth.core [None req-48ec74b8-6aab-4e2e-ad5c-625d25c26ccc None None] MFA Rules not processed for user `65da864252a347a8a0c65f15597d2e8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:17:01.862277 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-48ec74b8-6aab-4e2e-ad5c-625d25c26ccc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:17:01.944696 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1142/2308] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:17:01 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:17:01.950026 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94cc877-00f2-4912-af02-4e41e36415f4 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:17:02.149907 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:17:02.157943 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94cc877-00f2-4912-af02-4e41e36415f4 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:17:02.163378 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d7c902-8745-4078-863b-4e93fb8002a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:17:02.164039 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d7c902-8745-4078-863b-4e93fb8002a8 admin admin] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:17:02.164325 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d7c902-8745-4078-863b-4e93fb8002a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:17:02.164598 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d7c902-8745-4078-863b-4e93fb8002a8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:17:02.165284 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d7c902-8745-4078-863b-4e93fb8002a8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:17:02.167408 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d7c902-8745-4078-863b-4e93fb8002a8 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:17:02.298920 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1167/2309] 10.222.0.46 () {60 vars in 1153 bytes} [Mon Oct 10 12:17:01 2022] POST /identity/v3/projects => generated 373 bytes in 350 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 12:17:02.303774 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ec74b8-6aab-4e2e-ad5c-625d25c26ccc None None] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:17:02.317452 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ec74b8-6aab-4e2e-ad5c-625d25c26ccc None None] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:17:02.381668 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0052ff-81d5-4ac7-92b7-870a61a87ffc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:17:02.382280 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0052ff-81d5-4ac7-92b7-870a61a87ffc admin admin] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:17:02.382559 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0052ff-81d5-4ac7-92b7-870a61a87ffc admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:17:02.382858 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0052ff-81d5-4ac7-92b7-870a61a87ffc admin admin] PATH_INFO: `/v3/users` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:17:02.383488 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0052ff-81d5-4ac7-92b7-870a61a87ffc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:17:02.385473 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0052ff-81d5-4ac7-92b7-870a61a87ffc admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:17:02.488754 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1143/2310] 10.222.0.46 () {60 vars in 1144 bytes} [Mon Oct 10 12:17:02 2022] POST /identity/v3/users => generated 293 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 12:17:02.494354 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d7c902-8745-4078-863b-4e93fb8002a8 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:17:02.505862 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d7c902-8745-4078-863b-4e93fb8002a8 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:17:02.510934 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93683644-5003-4ae7-a2c3-582d6bf0d4a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:17:02.512065 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93683644-5003-4ae7-a2c3-582d6bf0d4a7 admin admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:17:02.512351 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93683644-5003-4ae7-a2c3-582d6bf0d4a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:17:02.512619 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93683644-5003-4ae7-a2c3-582d6bf0d4a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:17:02.513272 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93683644-5003-4ae7-a2c3-582d6bf0d4a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:17:02.514991 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93683644-5003-4ae7-a2c3-582d6bf0d4a7 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:17:02.527593 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1168/2311] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 10 12:17:02 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 12:17:02.532191 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0052ff-81d5-4ac7-92b7-870a61a87ffc admin admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:17:02.543540 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0052ff-81d5-4ac7-92b7-870a61a87ffc admin admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:17:02.548749 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d3b51d-1090-4154-ba65-ef0e6595e569 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:17:02.553168 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d3b51d-1090-4154-ba65-ef0e6595e569 admin admin] REQUEST_METHOD: `PUT` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:17:02.553168 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d3b51d-1090-4154-ba65-ef0e6595e569 admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:17:02.553168 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d3b51d-1090-4154-ba65-ef0e6595e569 admin admin] PATH_INFO: `/v3/projects/da9cbeb3d5c64c2b9d43c0540664d52d/users/cdce1dd1ab3f4605b31f713ac07ad68f/roles/0d822a0674b6495fac623877a3b38025` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:17:02.566662 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d3b51d-1090-4154-ba65-ef0e6595e569 admin admin] RBAC: Authorizing `identity:create_grant(project_id=da9cbeb3d5c64c2b9d43c0540664d52d, user_id=cdce1dd1ab3f4605b31f713ac07ad68f, role_id=0d822a0674b6495fac623877a3b38025)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:17:02.569342 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d3b51d-1090-4154-ba65-ef0e6595e569 admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:17:02.645223 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1144/2312] 10.222.0.46 () {60 vars in 1483 bytes} [Mon Oct 10 12:17:02 2022] PUT /identity/v3/projects/da9cbeb3d5c64c2b9d43c0540664d52d/users/cdce1dd1ab3f4605b31f713ac07ad68f/roles/0d822a0674b6495fac623877a3b38025 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 10 12:17:02.650247 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93683644-5003-4ae7-a2c3-582d6bf0d4a7 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:17:02.661739 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93683644-5003-4ae7-a2c3-582d6bf0d4a7 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:17:02.725313 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07a3108-532d-487c-b963-23c85a8061d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:17:02.725950 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07a3108-532d-487c-b963-23c85a8061d2 admin admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:17:02.726261 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07a3108-532d-487c-b963-23c85a8061d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:17:02.726564 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07a3108-532d-487c-b963-23c85a8061d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:17:02.727265 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07a3108-532d-487c-b963-23c85a8061d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:17:02.729029 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07a3108-532d-487c-b963-23c85a8061d2 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:17:02.740316 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1169/2313] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 10 12:17:02 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 12:17:02.745662 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d3b51d-1090-4154-ba65-ef0e6595e569 admin admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:17:02.756892 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d3b51d-1090-4154-ba65-ef0e6595e569 admin admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:17:02.769472 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ed3788-9544-4103-9dad-339e50924a39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:17:02.770244 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ed3788-9544-4103-9dad-339e50924a39 admin admin] REQUEST_METHOD: `PUT` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:17:02.770537 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ed3788-9544-4103-9dad-339e50924a39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:17:02.770889 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ed3788-9544-4103-9dad-339e50924a39 admin admin] PATH_INFO: `/v3/projects/da9cbeb3d5c64c2b9d43c0540664d52d/users/cdce1dd1ab3f4605b31f713ac07ad68f/roles/0d822a0674b6495fac623877a3b38025` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:17:02.772939 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ed3788-9544-4103-9dad-339e50924a39 admin admin] RBAC: Authorizing `identity:create_grant(project_id=da9cbeb3d5c64c2b9d43c0540664d52d, user_id=cdce1dd1ab3f4605b31f713ac07ad68f, role_id=0d822a0674b6495fac623877a3b38025)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:17:02.775882 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ed3788-9544-4103-9dad-339e50924a39 admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:17:02.780257 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:17:02.788878 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1145/2314] 10.222.0.46 () {60 vars in 1483 bytes} [Mon Oct 10 12:17:02 2022] PUT /identity/v3/projects/da9cbeb3d5c64c2b9d43c0540664d52d/users/cdce1dd1ab3f4605b31f713ac07ad68f/roles/0d822a0674b6495fac623877a3b38025 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:17:02.807325 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0adb01d-359c-44cc-af72-8dcc4aed27d8 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:17:02.807813 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0adb01d-359c-44cc-af72-8dcc4aed27d8 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:17:02.808303 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0adb01d-359c-44cc-af72-8dcc4aed27d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:17:02.855783 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-f0adb01d-359c-44cc-af72-8dcc4aed27d8 None None] MFA Rules not processed for user `cdce1dd1ab3f4605b31f713ac07ad68f`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:17:02.857218 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-f0adb01d-359c-44cc-af72-8dcc4aed27d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:17:02.949179 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1170/2315] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:17:02 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:17:02.965375 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ed3788-9544-4103-9dad-339e50924a39 admin admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:17:02.967116 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-c3ed3788-9544-4103-9dad-339e50924a39 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:17:03.034208 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ed3788-9544-4103-9dad-339e50924a39 admin admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:17:03.038058 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f21dae4-2c62-446c-bfad-983a54ade17c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3ae093470484b93b23dcfae1657df9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad3772f7b6a47c28b7ba37a9316a3c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:17:03.038742 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f21dae4-2c62-446c-bfad-983a54ade17c service cinder] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:17:03.039023 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f21dae4-2c62-446c-bfad-983a54ade17c service cinder] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:17:03.039299 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f21dae4-2c62-446c-bfad-983a54ade17c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:17:03.049318 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f21dae4-2c62-446c-bfad-983a54ade17c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:17:03.050857 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f21dae4-2c62-446c-bfad-983a54ade17c service cinder] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:17:03.061973 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1146/2316] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 10 12:17:02 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:17:03.577416 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0adb01d-359c-44cc-af72-8dcc4aed27d8 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:17:03.588026 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0adb01d-359c-44cc-af72-8dcc4aed27d8 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:17:03.592661 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f648c423-383d-497d-bae5-2e8f56dea97e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3ae093470484b93b23dcfae1657df9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad3772f7b6a47c28b7ba37a9316a3c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:17:03.593270 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f648c423-383d-497d-bae5-2e8f56dea97e service cinder] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:17:03.593551 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f648c423-383d-497d-bae5-2e8f56dea97e service cinder] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:17:03.593825 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f648c423-383d-497d-bae5-2e8f56dea97e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:17:03.648908 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:17:03.659269 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f648c423-383d-497d-bae5-2e8f56dea97e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:17:03.660735 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f648c423-383d-497d-bae5-2e8f56dea97e service cinder] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:17:03.670366 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1171/2317] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 10 12:17:03 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:17:13.056789 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=28 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:17:13.952342 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=5 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:18:04.621545 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f21dae4-2c62-446c-bfad-983a54ade17c service cinder] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:18:04.682465 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f21dae4-2c62-446c-bfad-983a54ade17c service cinder] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:18:04.687053 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7fdcc12-9c5c-4c97-9fe6-64ee28993d8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:18:04.687680 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fdcc12-9c5c-4c97-9fe6-64ee28993d8e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:18:04.687968 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fdcc12-9c5c-4c97-9fe6-64ee28993d8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:18:04.688238 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fdcc12-9c5c-4c97-9fe6-64ee28993d8e admin admin] PATH_INFO: `/v3/users/65da864252a347a8a0c65f15597d2e8b` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:18:04.689360 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7fdcc12-9c5c-4c97-9fe6-64ee28993d8e admin admin] RBAC: Authorizing `identity:delete_user(user_id=65da864252a347a8a0c65f15597d2e8b)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:18:04.691312 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7fdcc12-9c5c-4c97-9fe6-64ee28993d8e admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:18:04.896005 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-c7fdcc12-9c5c-4c97-9fe6-64ee28993d8e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '65da864252a347a8a0c65f15597d2e8b'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:18:04.902960 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-c7fdcc12-9c5c-4c97-9fe6-64ee28993d8e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '65da864252a347a8a0c65f15597d2e8b'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:18:04.903646 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-c7fdcc12-9c5c-4c97-9fe6-64ee28993d8e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '65da864252a347a8a0c65f15597d2e8b'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:18:04.991123 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-c7fdcc12-9c5c-4c97-9fe6-64ee28993d8e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '65da864252a347a8a0c65f15597d2e8b'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:18:05.004419 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-c7fdcc12-9c5c-4c97-9fe6-64ee28993d8e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '65da864252a347a8a0c65f15597d2e8b'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:18:05.010387 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1147/2318] 10.222.0.46 () {58 vars in 1224 bytes} [Mon Oct 10 12:18:04 2022] DELETE /identity/v3/users/65da864252a347a8a0c65f15597d2e8b => generated 0 bytes in 389 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:18:05.014981 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddb0fe3-3460-4c48-9708-c6745d78f6f6 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:18:05.015453 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddb0fe3-3460-4c48-9708-c6745d78f6f6 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:18:05.015625 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddb0fe3-3460-4c48-9708-c6745d78f6f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:18:05.036509 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-5ddb0fe3-3460-4c48-9708-c6745d78f6f6 None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:18:05.037724 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-5ddb0fe3-3460-4c48-9708-c6745d78f6f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:18:05.099285 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1172/2319] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:18:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:18:05.111433 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7fdcc12-9c5c-4c97-9fe6-64ee28993d8e admin admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:18:05.113007 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-c7fdcc12-9c5c-4c97-9fe6-64ee28993d8e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:18:05.171560 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7fdcc12-9c5c-4c97-9fe6-64ee28993d8e admin admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:18:05.175437 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4409072-ee2f-4ba3-aba7-a5f0a280b46e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58b1c7c85302421cba823ace4201861f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad3772f7b6a47c28b7ba37a9316a3c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:18:05.176042 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4409072-ee2f-4ba3-aba7-a5f0a280b46e service neutron] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:18:05.176327 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4409072-ee2f-4ba3-aba7-a5f0a280b46e service neutron] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:18:05.176605 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4409072-ee2f-4ba3-aba7-a5f0a280b46e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:18:05.185921 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4409072-ee2f-4ba3-aba7-a5f0a280b46e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:18:05.187395 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4409072-ee2f-4ba3-aba7-a5f0a280b46e service neutron] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:18:05.197577 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1148/2320] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 10 12:18:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:18:05.610202 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ddb0fe3-3460-4c48-9708-c6745d78f6f6 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:18:05.611748 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-5ddb0fe3-3460-4c48-9708-c6745d78f6f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:18:05.628913 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ddb0fe3-3460-4c48-9708-c6745d78f6f6 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:18:05.635351 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877284cc-5e49-4f4b-91fd-a188f265fb25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:18:05.636011 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877284cc-5e49-4f4b-91fd-a188f265fb25 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:18:05.636317 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877284cc-5e49-4f4b-91fd-a188f265fb25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:18:05.636820 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877284cc-5e49-4f4b-91fd-a188f265fb25 admin admin] PATH_INFO: `/v3/projects/3ffdb088d2f7493996841f5ef0d4b612` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:18:05.638483 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877284cc-5e49-4f4b-91fd-a188f265fb25 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3ffdb088d2f7493996841f5ef0d4b612)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:18:05.640301 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877284cc-5e49-4f4b-91fd-a188f265fb25 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:18:05.770328 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-877284cc-5e49-4f4b-91fd-a188f265fb25 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3ffdb088d2f7493996841f5ef0d4b612', 'request_id': 'req-877284cc-5e49-4f4b-91fd-a188f265fb25'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:18:05.776833 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-877284cc-5e49-4f4b-91fd-a188f265fb25 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3ffdb088d2f7493996841f5ef0d4b612', 'request_id': 'req-877284cc-5e49-4f4b-91fd-a188f265fb25'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:18:05.783384 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-877284cc-5e49-4f4b-91fd-a188f265fb25 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3ffdb088d2f7493996841f5ef0d4b612', 'request_id': 'req-877284cc-5e49-4f4b-91fd-a188f265fb25'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:18:05.870611 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-877284cc-5e49-4f4b-91fd-a188f265fb25 admin admin] The token cache is being invalidate because project 3ffdb088d2f7493996841f5ef0d4b612 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81677) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 10 12:18:05.870929 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-877284cc-5e49-4f4b-91fd-a188f265fb25 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:18:05.871545 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-877284cc-5e49-4f4b-91fd-a188f265fb25 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:18:05.873067 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1173/2321] 10.222.0.46 () {58 vars in 1233 bytes} [Mon Oct 10 12:18:05 2022] DELETE /identity/v3/projects/3ffdb088d2f7493996841f5ef0d4b612 => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:18:05.879038 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4409072-ee2f-4ba3-aba7-a5f0a280b46e service neutron] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:18:05.888676 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4409072-ee2f-4ba3-aba7-a5f0a280b46e service neutron] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:18:05.906293 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:18:05.944841 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a81e54-95b7-4e5c-8511-dea886e3c6b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:18:05.945462 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a81e54-95b7-4e5c-8511-dea886e3c6b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:18:05.945739 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a81e54-95b7-4e5c-8511-dea886e3c6b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:18:05.946011 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a81e54-95b7-4e5c-8511-dea886e3c6b3 admin admin] PATH_INFO: `/v3/users/cdce1dd1ab3f4605b31f713ac07ad68f` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:18:05.947150 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a81e54-95b7-4e5c-8511-dea886e3c6b3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cdce1dd1ab3f4605b31f713ac07ad68f)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:18:05.949048 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a81e54-95b7-4e5c-8511-dea886e3c6b3 admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:18:06.182175 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-a0a81e54-95b7-4e5c-8511-dea886e3c6b3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cdce1dd1ab3f4605b31f713ac07ad68f'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:18:06.189442 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-a0a81e54-95b7-4e5c-8511-dea886e3c6b3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cdce1dd1ab3f4605b31f713ac07ad68f'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:18:06.190156 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-a0a81e54-95b7-4e5c-8511-dea886e3c6b3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cdce1dd1ab3f4605b31f713ac07ad68f'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:18:06.294263 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-a0a81e54-95b7-4e5c-8511-dea886e3c6b3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cdce1dd1ab3f4605b31f713ac07ad68f'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:18:06.307409 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-a0a81e54-95b7-4e5c-8511-dea886e3c6b3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cdce1dd1ab3f4605b31f713ac07ad68f'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:18:06.313864 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1149/2322] 10.222.0.46 () {58 vars in 1224 bytes} [Mon Oct 10 12:18:05 2022] DELETE /identity/v3/users/cdce1dd1ab3f4605b31f713ac07ad68f => generated 0 bytes in 435 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 10 12:18:06.774961 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877284cc-5e49-4f4b-91fd-a188f265fb25 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:18:06.776415 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-877284cc-5e49-4f4b-91fd-a188f265fb25 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:18:06.789583 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877284cc-5e49-4f4b-91fd-a188f265fb25 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:18:06.851738 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5cebfc-4bee-4db4-8b2b-e43a335e626d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:18:06.852366 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5cebfc-4bee-4db4-8b2b-e43a335e626d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:18:06.852643 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5cebfc-4bee-4db4-8b2b-e43a335e626d admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:18:06.852917 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5cebfc-4bee-4db4-8b2b-e43a335e626d admin admin] PATH_INFO: `/v3/projects/da9cbeb3d5c64c2b9d43c0540664d52d` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:18:06.854025 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5cebfc-4bee-4db4-8b2b-e43a335e626d admin admin] RBAC: Authorizing `identity:delete_project(project_id=da9cbeb3d5c64c2b9d43c0540664d52d)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:18:06.855946 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5cebfc-4bee-4db4-8b2b-e43a335e626d admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:18:06.958270 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:18:06.974841 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-2f5cebfc-4bee-4db4-8b2b-e43a335e626d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'da9cbeb3d5c64c2b9d43c0540664d52d', 'request_id': 'req-2f5cebfc-4bee-4db4-8b2b-e43a335e626d'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:18:06.980435 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-2f5cebfc-4bee-4db4-8b2b-e43a335e626d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'da9cbeb3d5c64c2b9d43c0540664d52d', 'request_id': 'req-2f5cebfc-4bee-4db4-8b2b-e43a335e626d'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:18:06.986568 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-2f5cebfc-4bee-4db4-8b2b-e43a335e626d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'da9cbeb3d5c64c2b9d43c0540664d52d', 'request_id': 'req-2f5cebfc-4bee-4db4-8b2b-e43a335e626d'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:18:07.076931 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-2f5cebfc-4bee-4db4-8b2b-e43a335e626d admin admin] The token cache is being invalidate because project da9cbeb3d5c64c2b9d43c0540664d52d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81677) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 10 12:18:07.077134 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-2f5cebfc-4bee-4db4-8b2b-e43a335e626d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:18:07.077727 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-2f5cebfc-4bee-4db4-8b2b-e43a335e626d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:18:07.079383 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1174/2323] 10.222.0.46 () {58 vars in 1233 bytes} [Mon Oct 10 12:18:06 2022] DELETE /identity/v3/projects/da9cbeb3d5c64c2b9d43c0540664d52d => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:18:07.084338 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70a98ab-7add-4b28-ad80-7ca70cd8e691 None None] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:18:07.084813 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70a98ab-7add-4b28-ad80-7ca70cd8e691 None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:18:07.084927 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70a98ab-7add-4b28-ad80-7ca70cd8e691 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:18:07.105750 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.auth.core [None req-c70a98ab-7add-4b28-ad80-7ca70cd8e691 None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:18:07.107013 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-c70a98ab-7add-4b28-ad80-7ca70cd8e691 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:18:07.168260 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1150/2324] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:18:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:18:07.177760 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2787b14f-cf7e-4fab-9b6d-275b07dfffbd None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:18:07.178082 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2787b14f-cf7e-4fab-9b6d-275b07dfffbd None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:18:07.178360 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2787b14f-cf7e-4fab-9b6d-275b07dfffbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:18:07.197616 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-2787b14f-cf7e-4fab-9b6d-275b07dfffbd None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:18:07.198869 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-2787b14f-cf7e-4fab-9b6d-275b07dfffbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:18:07.206629 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1175/2325] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:18:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:18:07.211002 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70a98ab-7add-4b28-ad80-7ca70cd8e691 None None] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:18:07.221283 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70a98ab-7add-4b28-ad80-7ca70cd8e691 None None] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:18:07.226529 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c77d5cb-fbd0-426b-872f-7be8f9c448d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:18:07.227165 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c77d5cb-fbd0-426b-872f-7be8f9c448d9 admin admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:18:07.227450 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c77d5cb-fbd0-426b-872f-7be8f9c448d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:18:07.227727 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c77d5cb-fbd0-426b-872f-7be8f9c448d9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:18:07.228380 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c77d5cb-fbd0-426b-872f-7be8f9c448d9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 12:18:07.228688 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c77d5cb-fbd0-426b-872f-7be8f9c448d9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:18:07.230339 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c77d5cb-fbd0-426b-872f-7be8f9c448d9 admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:18:07.243001 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1151/2326] 10.222.0.46 () {58 vars in 1153 bytes} [Mon Oct 10 12:18:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 12:18:07.247233 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2787b14f-cf7e-4fab-9b6d-275b07dfffbd None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:18:07.256316 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2787b14f-cf7e-4fab-9b6d-275b07dfffbd None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:18:07.260945 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6789b246-1030-4b1d-8f3f-6da601d3a2d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:18:07.261533 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6789b246-1030-4b1d-8f3f-6da601d3a2d7 admin admin] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:18:07.261811 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6789b246-1030-4b1d-8f3f-6da601d3a2d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:18:07.262081 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6789b246-1030-4b1d-8f3f-6da601d3a2d7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:18:07.262796 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6789b246-1030-4b1d-8f3f-6da601d3a2d7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:18:07.264677 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6789b246-1030-4b1d-8f3f-6da601d3a2d7 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:18:07.358311 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1176/2327] 10.222.0.46 () {60 vars in 1153 bytes} [Mon Oct 10 12:18:07 2022] POST /identity/v3/projects => generated 365 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 12:18:07.362459 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c77d5cb-fbd0-426b-872f-7be8f9c448d9 admin admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:18:07.373026 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c77d5cb-fbd0-426b-872f-7be8f9c448d9 admin admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:18:07.398347 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:18:07.430345 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfec6204-a983-4e37-b6b9-5438361dd0a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:18:07.430979 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfec6204-a983-4e37-b6b9-5438361dd0a4 admin admin] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:18:07.431265 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfec6204-a983-4e37-b6b9-5438361dd0a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:18:07.431542 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfec6204-a983-4e37-b6b9-5438361dd0a4 admin admin] PATH_INFO: `/v3/users` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:18:07.432187 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfec6204-a983-4e37-b6b9-5438361dd0a4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:18:07.434172 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfec6204-a983-4e37-b6b9-5438361dd0a4 admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:18:07.544509 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1152/2328] 10.222.0.46 () {60 vars in 1144 bytes} [Mon Oct 10 12:18:07 2022] POST /identity/v3/users => generated 283 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 12:18:07.548645 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6789b246-1030-4b1d-8f3f-6da601d3a2d7 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:18:07.559155 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6789b246-1030-4b1d-8f3f-6da601d3a2d7 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:18:07.563888 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4993d6-0758-477d-b033-93412b9076e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:18:07.564483 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4993d6-0758-477d-b033-93412b9076e2 admin admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:18:07.564759 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4993d6-0758-477d-b033-93412b9076e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:18:07.565029 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4993d6-0758-477d-b033-93412b9076e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:18:07.565704 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4993d6-0758-477d-b033-93412b9076e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:18:07.567350 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4993d6-0758-477d-b033-93412b9076e2 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:18:07.577146 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1177/2329] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 10 12:18:07 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 12:18:07.585838 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfec6204-a983-4e37-b6b9-5438361dd0a4 admin admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:18:07.596480 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfec6204-a983-4e37-b6b9-5438361dd0a4 admin admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:18:07.601940 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808eb155-c194-43cc-a82b-04f73dcd7ea9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:18:07.602535 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808eb155-c194-43cc-a82b-04f73dcd7ea9 admin admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:18:07.602852 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808eb155-c194-43cc-a82b-04f73dcd7ea9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:18:07.603132 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808eb155-c194-43cc-a82b-04f73dcd7ea9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:18:07.603779 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808eb155-c194-43cc-a82b-04f73dcd7ea9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:18:07.605400 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808eb155-c194-43cc-a82b-04f73dcd7ea9 admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:18:07.616132 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1153/2330] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 10 12:18:07 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 12:18:07.620281 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4993d6-0758-477d-b033-93412b9076e2 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:18:07.630447 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4993d6-0758-477d-b033-93412b9076e2 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:18:07.635262 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdaa40bc-c6a7-4c2c-b34f-fb6817b092c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:18:07.635908 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdaa40bc-c6a7-4c2c-b34f-fb6817b092c9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:18:07.636193 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdaa40bc-c6a7-4c2c-b34f-fb6817b092c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:18:07.636470 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdaa40bc-c6a7-4c2c-b34f-fb6817b092c9 admin admin] PATH_INFO: `/v3/projects/621850915a4a4101bfff21c5e2dd0f44/users/71e7c8e61eb14b9291c2f9fdef85cf7d/roles/b2cb2a6e818e47338ce071e6fedfc673` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:18:07.651801 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdaa40bc-c6a7-4c2c-b34f-fb6817b092c9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=621850915a4a4101bfff21c5e2dd0f44, user_id=71e7c8e61eb14b9291c2f9fdef85cf7d, role_id=b2cb2a6e818e47338ce071e6fedfc673)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:18:07.657635 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdaa40bc-c6a7-4c2c-b34f-fb6817b092c9 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:18:07.745271 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1178/2331] 10.222.0.46 () {60 vars in 1483 bytes} [Mon Oct 10 12:18:07 2022] PUT /identity/v3/projects/621850915a4a4101bfff21c5e2dd0f44/users/71e7c8e61eb14b9291c2f9fdef85cf7d/roles/b2cb2a6e818e47338ce071e6fedfc673 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:18:07.759014 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef961ed-00a4-4561-97de-d359b3255946 None None] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:18:07.759302 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef961ed-00a4-4561-97de-d359b3255946 None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:18:07.763644 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef961ed-00a4-4561-97de-d359b3255946 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:18:07.788921 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.auth.core [None req-8ef961ed-00a4-4561-97de-d359b3255946 None None] MFA Rules not processed for user `71e7c8e61eb14b9291c2f9fdef85cf7d`. Rule list: `[]` (Enabled: `True`). {{(pid=81676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:18:07.790151 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-8ef961ed-00a4-4561-97de-d359b3255946 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:18:07.854236 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1154/2332] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:18:07 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:18:07.858728 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdaa40bc-c6a7-4c2c-b34f-fb6817b092c9 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:18:07.919581 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdaa40bc-c6a7-4c2c-b34f-fb6817b092c9 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:18:07.924401 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308c871e-8ff4-4369-8cdc-a03bb8b4b0b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:18:07.925013 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308c871e-8ff4-4369-8cdc-a03bb8b4b0b8 admin admin] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:18:07.925293 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308c871e-8ff4-4369-8cdc-a03bb8b4b0b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:18:07.925569 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308c871e-8ff4-4369-8cdc-a03bb8b4b0b8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:18:07.926196 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308c871e-8ff4-4369-8cdc-a03bb8b4b0b8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:18:07.928129 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308c871e-8ff4-4369-8cdc-a03bb8b4b0b8 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:18:08.071834 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1179/2333] 10.222.0.46 () {60 vars in 1153 bytes} [Mon Oct 10 12:18:07 2022] POST /identity/v3/projects => generated 363 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 12:18:08.075962 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef961ed-00a4-4561-97de-d359b3255946 None None] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:18:08.087425 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef961ed-00a4-4561-97de-d359b3255946 None None] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:18:08.144897 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75761985-249c-445c-b7ce-399ed16bf8a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:18:08.145543 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75761985-249c-445c-b7ce-399ed16bf8a1 admin admin] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:18:08.145828 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75761985-249c-445c-b7ce-399ed16bf8a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:18:08.146107 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75761985-249c-445c-b7ce-399ed16bf8a1 admin admin] PATH_INFO: `/v3/users` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:18:08.146783 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75761985-249c-445c-b7ce-399ed16bf8a1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:18:08.148701 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75761985-249c-445c-b7ce-399ed16bf8a1 admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:18:08.237099 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1155/2334] 10.222.0.46 () {60 vars in 1144 bytes} [Mon Oct 10 12:18:08 2022] POST /identity/v3/users => generated 288 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 12:18:08.241638 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308c871e-8ff4-4369-8cdc-a03bb8b4b0b8 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:18:08.251883 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308c871e-8ff4-4369-8cdc-a03bb8b4b0b8 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:18:08.257109 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab250d8-d036-44f8-b837-0f13fad92d9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:18:08.257713 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab250d8-d036-44f8-b837-0f13fad92d9c admin admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:18:08.257990 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab250d8-d036-44f8-b837-0f13fad92d9c admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:18:08.258263 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab250d8-d036-44f8-b837-0f13fad92d9c admin admin] PATH_INFO: `/v3/roles` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:18:08.258939 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab250d8-d036-44f8-b837-0f13fad92d9c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:18:08.260530 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab250d8-d036-44f8-b837-0f13fad92d9c admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:18:08.271307 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1180/2335] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 10 12:18:08 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 12:18:08.275486 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75761985-249c-445c-b7ce-399ed16bf8a1 admin admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:18:08.285865 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75761985-249c-445c-b7ce-399ed16bf8a1 admin admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:18:08.290481 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ede647-a63e-45e3-a8df-286eafd34da5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:18:08.291172 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ede647-a63e-45e3-a8df-286eafd34da5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:18:08.291456 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ede647-a63e-45e3-a8df-286eafd34da5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:18:08.291733 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ede647-a63e-45e3-a8df-286eafd34da5 admin admin] PATH_INFO: `/v3/projects/a406f08c603e4bdcb7596d6b1d59a8e5/users/c7d94693bb4c4c4fae81b980ed6b96be/roles/0d822a0674b6495fac623877a3b38025` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:18:08.307245 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ede647-a63e-45e3-a8df-286eafd34da5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a406f08c603e4bdcb7596d6b1d59a8e5, user_id=c7d94693bb4c4c4fae81b980ed6b96be, role_id=0d822a0674b6495fac623877a3b38025)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:18:08.309996 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ede647-a63e-45e3-a8df-286eafd34da5 admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:18:08.415147 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1156/2336] 10.222.0.46 () {60 vars in 1483 bytes} [Mon Oct 10 12:18:08 2022] PUT /identity/v3/projects/a406f08c603e4bdcb7596d6b1d59a8e5/users/c7d94693bb4c4c4fae81b980ed6b96be/roles/0d822a0674b6495fac623877a3b38025 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 10 12:18:08.419502 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab250d8-d036-44f8-b837-0f13fad92d9c admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:18:08.430555 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab250d8-d036-44f8-b837-0f13fad92d9c admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:18:08.451243 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:18:08.490590 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42876674-fa2e-4d00-863a-04ccfc72b0ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:18:08.497285 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42876674-fa2e-4d00-863a-04ccfc72b0ec admin admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:18:08.497285 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42876674-fa2e-4d00-863a-04ccfc72b0ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:18:08.497285 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42876674-fa2e-4d00-863a-04ccfc72b0ec admin admin] PATH_INFO: `/v3/roles` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:18:08.497285 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42876674-fa2e-4d00-863a-04ccfc72b0ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:18:08.497285 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42876674-fa2e-4d00-863a-04ccfc72b0ec admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:18:08.504179 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1181/2337] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 10 12:18:08 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 10 12:18:08.508679 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ede647-a63e-45e3-a8df-286eafd34da5 admin admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:18:08.514188 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:18:08.519729 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ede647-a63e-45e3-a8df-286eafd34da5 admin admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:18:08.524493 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d003c4-49c1-437c-b7b7-995ae8c7c1cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:18:08.531755 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d003c4-49c1-437c-b7b7-995ae8c7c1cc admin admin] REQUEST_METHOD: `PUT` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:18:08.531755 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d003c4-49c1-437c-b7b7-995ae8c7c1cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:18:08.531755 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d003c4-49c1-437c-b7b7-995ae8c7c1cc admin admin] PATH_INFO: `/v3/projects/a406f08c603e4bdcb7596d6b1d59a8e5/users/c7d94693bb4c4c4fae81b980ed6b96be/roles/0d822a0674b6495fac623877a3b38025` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:18:08.531755 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d003c4-49c1-437c-b7b7-995ae8c7c1cc admin admin] RBAC: Authorizing `identity:create_grant(project_id=a406f08c603e4bdcb7596d6b1d59a8e5, user_id=c7d94693bb4c4c4fae81b980ed6b96be, role_id=0d822a0674b6495fac623877a3b38025)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:18:08.531755 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d003c4-49c1-437c-b7b7-995ae8c7c1cc admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:18:08.540849 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1157/2338] 10.222.0.46 () {60 vars in 1483 bytes} [Mon Oct 10 12:18:08 2022] PUT /identity/v3/projects/a406f08c603e4bdcb7596d6b1d59a8e5/users/c7d94693bb4c4c4fae81b980ed6b96be/roles/0d822a0674b6495fac623877a3b38025 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:18:08.552613 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b96cf29-cedb-474e-bfa3-6c0b36abc2a5 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:18:08.552902 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b96cf29-cedb-474e-bfa3-6c0b36abc2a5 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:18:08.553182 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b96cf29-cedb-474e-bfa3-6c0b36abc2a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:18:08.580764 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-6b96cf29-cedb-474e-bfa3-6c0b36abc2a5 None None] MFA Rules not processed for user `c7d94693bb4c4c4fae81b980ed6b96be`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:18:08.581998 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-6b96cf29-cedb-474e-bfa3-6c0b36abc2a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:18:08.648991 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1182/2339] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:18:08 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:18:08.661276 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d003c4-49c1-437c-b7b7-995ae8c7c1cc admin admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:18:08.665706 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-84d003c4-49c1-437c-b7b7-995ae8c7c1cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:18:08.720205 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d003c4-49c1-437c-b7b7-995ae8c7c1cc admin admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:18:08.724445 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a750f8-c090-4f49-9b7c-7dee738753cc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3ae093470484b93b23dcfae1657df9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad3772f7b6a47c28b7ba37a9316a3c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:18:08.725053 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a750f8-c090-4f49-9b7c-7dee738753cc service cinder] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:18:08.725333 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a750f8-c090-4f49-9b7c-7dee738753cc service cinder] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:18:08.725607 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a750f8-c090-4f49-9b7c-7dee738753cc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:18:08.734779 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a750f8-c090-4f49-9b7c-7dee738753cc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:18:08.736276 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a750f8-c090-4f49-9b7c-7dee738753cc service cinder] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:18:08.746776 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1158/2340] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 10 12:18:08 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:18:09.745403 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b96cf29-cedb-474e-bfa3-6c0b36abc2a5 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:18:09.810250 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b96cf29-cedb-474e-bfa3-6c0b36abc2a5 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:18:09.815102 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16fe642-945d-4fe2-a4eb-5ef7bd17b4ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:18:09.815785 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16fe642-945d-4fe2-a4eb-5ef7bd17b4ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:18:09.816593 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16fe642-945d-4fe2-a4eb-5ef7bd17b4ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:18:09.816868 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16fe642-945d-4fe2-a4eb-5ef7bd17b4ec admin admin] PATH_INFO: `/v3/users/71e7c8e61eb14b9291c2f9fdef85cf7d` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:18:09.818074 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16fe642-945d-4fe2-a4eb-5ef7bd17b4ec admin admin] RBAC: Authorizing `identity:delete_user(user_id=71e7c8e61eb14b9291c2f9fdef85cf7d)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:18:09.820020 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16fe642-945d-4fe2-a4eb-5ef7bd17b4ec admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:18:10.028361 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-b16fe642-945d-4fe2-a4eb-5ef7bd17b4ec admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '71e7c8e61eb14b9291c2f9fdef85cf7d'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:18:10.029654 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:18:10.036425 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-b16fe642-945d-4fe2-a4eb-5ef7bd17b4ec admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '71e7c8e61eb14b9291c2f9fdef85cf7d'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:18:10.037129 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-b16fe642-945d-4fe2-a4eb-5ef7bd17b4ec admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '71e7c8e61eb14b9291c2f9fdef85cf7d'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:18:10.139838 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-b16fe642-945d-4fe2-a4eb-5ef7bd17b4ec admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '71e7c8e61eb14b9291c2f9fdef85cf7d'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:18:10.152721 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-b16fe642-945d-4fe2-a4eb-5ef7bd17b4ec admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '71e7c8e61eb14b9291c2f9fdef85cf7d'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:18:10.159297 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1183/2341] 10.222.0.46 () {58 vars in 1224 bytes} [Mon Oct 10 12:18:09 2022] DELETE /identity/v3/users/71e7c8e61eb14b9291c2f9fdef85cf7d => generated 0 bytes in 414 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:18:10.164301 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5a5071-8ef5-479c-9be1-a606c616558e None None] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:18:10.172886 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5a5071-8ef5-479c-9be1-a606c616558e None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:18:10.172886 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5a5071-8ef5-479c-9be1-a606c616558e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:18:10.184696 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.auth.core [None req-ba5a5071-8ef5-479c-9be1-a606c616558e None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:18:10.185913 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-ba5a5071-8ef5-479c-9be1-a606c616558e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:18:10.246104 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1159/2342] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:18:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 10 12:18:10.256023 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16fe642-945d-4fe2-a4eb-5ef7bd17b4ec admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:18:10.257408 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-b16fe642-945d-4fe2-a4eb-5ef7bd17b4ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:18:10.315792 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16fe642-945d-4fe2-a4eb-5ef7bd17b4ec admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:18:10.319309 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7acd2b92-b240-407b-9be2-b8e24c812cba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58b1c7c85302421cba823ace4201861f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad3772f7b6a47c28b7ba37a9316a3c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:18:10.319907 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acd2b92-b240-407b-9be2-b8e24c812cba service neutron] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:18:10.320178 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acd2b92-b240-407b-9be2-b8e24c812cba service neutron] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:18:10.320446 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acd2b92-b240-407b-9be2-b8e24c812cba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:18:10.329259 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7acd2b92-b240-407b-9be2-b8e24c812cba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:18:10.330658 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7acd2b92-b240-407b-9be2-b8e24c812cba service neutron] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:18:10.339875 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1184/2343] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 10 12:18:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:18:10.804256 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5a5071-8ef5-479c-9be1-a606c616558e None None] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:18:10.805748 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-ba5a5071-8ef5-479c-9be1-a606c616558e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:18:10.821347 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5a5071-8ef5-479c-9be1-a606c616558e None None] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:18:10.829183 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81452180-6cf4-490f-bc2a-87bf6b63966f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:18:10.829183 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81452180-6cf4-490f-bc2a-87bf6b63966f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:18:10.829183 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81452180-6cf4-490f-bc2a-87bf6b63966f admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:18:10.829183 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81452180-6cf4-490f-bc2a-87bf6b63966f admin admin] PATH_INFO: `/v3/projects/621850915a4a4101bfff21c5e2dd0f44` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:18:10.829784 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81452180-6cf4-490f-bc2a-87bf6b63966f admin admin] RBAC: Authorizing `identity:delete_project(project_id=621850915a4a4101bfff21c5e2dd0f44)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:18:10.832216 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81452180-6cf4-490f-bc2a-87bf6b63966f admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:18:10.939401 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-81452180-6cf4-490f-bc2a-87bf6b63966f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '621850915a4a4101bfff21c5e2dd0f44', 'request_id': 'req-81452180-6cf4-490f-bc2a-87bf6b63966f'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:18:10.945842 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-81452180-6cf4-490f-bc2a-87bf6b63966f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '621850915a4a4101bfff21c5e2dd0f44', 'request_id': 'req-81452180-6cf4-490f-bc2a-87bf6b63966f'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:18:10.952095 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-81452180-6cf4-490f-bc2a-87bf6b63966f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '621850915a4a4101bfff21c5e2dd0f44', 'request_id': 'req-81452180-6cf4-490f-bc2a-87bf6b63966f'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:18:11.042052 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-81452180-6cf4-490f-bc2a-87bf6b63966f admin admin] The token cache is being invalidate because project 621850915a4a4101bfff21c5e2dd0f44 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81676) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 10 12:18:11.042378 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-81452180-6cf4-490f-bc2a-87bf6b63966f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:18:11.043031 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-81452180-6cf4-490f-bc2a-87bf6b63966f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:18:11.044538 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1160/2344] 10.222.0.46 () {58 vars in 1233 bytes} [Mon Oct 10 12:18:10 2022] DELETE /identity/v3/projects/621850915a4a4101bfff21c5e2dd0f44 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:18:11.048636 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7acd2b92-b240-407b-9be2-b8e24c812cba service neutron] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:18:11.058280 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7acd2b92-b240-407b-9be2-b8e24c812cba service neutron] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:18:11.114248 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5370cce7-2b6c-4876-9407-3da204ed9c18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:18:11.114907 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5370cce7-2b6c-4876-9407-3da204ed9c18 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:18:11.115190 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5370cce7-2b6c-4876-9407-3da204ed9c18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:18:11.115464 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5370cce7-2b6c-4876-9407-3da204ed9c18 admin admin] PATH_INFO: `/v3/users/c7d94693bb4c4c4fae81b980ed6b96be` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:18:11.116581 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5370cce7-2b6c-4876-9407-3da204ed9c18 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c7d94693bb4c4c4fae81b980ed6b96be)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:18:11.118491 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5370cce7-2b6c-4876-9407-3da204ed9c18 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:18:11.325140 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-5370cce7-2b6c-4876-9407-3da204ed9c18 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c7d94693bb4c4c4fae81b980ed6b96be'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:18:11.332271 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-5370cce7-2b6c-4876-9407-3da204ed9c18 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c7d94693bb4c4c4fae81b980ed6b96be'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:18:11.332987 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-5370cce7-2b6c-4876-9407-3da204ed9c18 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c7d94693bb4c4c4fae81b980ed6b96be'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:18:11.413550 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-5370cce7-2b6c-4876-9407-3da204ed9c18 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c7d94693bb4c4c4fae81b980ed6b96be'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:18:11.419381 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=86,keystone:INSERT=2,keystone:DELETE=12 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:18:11.427352 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-5370cce7-2b6c-4876-9407-3da204ed9c18 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c7d94693bb4c4c4fae81b980ed6b96be'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:18:11.433998 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1185/2345] 10.222.0.46 () {58 vars in 1224 bytes} [Mon Oct 10 12:18:11 2022] DELETE /identity/v3/users/c7d94693bb4c4c4fae81b980ed6b96be => generated 0 bytes in 386 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:18:11.917225 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81452180-6cf4-490f-bc2a-87bf6b63966f admin admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:18:11.918822 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-81452180-6cf4-490f-bc2a-87bf6b63966f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:18:11.933152 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81452180-6cf4-490f-bc2a-87bf6b63966f admin admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:18:11.946329 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:18:11.994527 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695e4afb-acfe-4cae-b14a-5576ce060b8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:18:11.995334 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695e4afb-acfe-4cae-b14a-5576ce060b8c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:18:11.995582 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695e4afb-acfe-4cae-b14a-5576ce060b8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:18:11.995854 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695e4afb-acfe-4cae-b14a-5576ce060b8c admin admin] PATH_INFO: `/v3/projects/a406f08c603e4bdcb7596d6b1d59a8e5` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:18:11.997001 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695e4afb-acfe-4cae-b14a-5576ce060b8c admin admin] RBAC: Authorizing `identity:delete_project(project_id=a406f08c603e4bdcb7596d6b1d59a8e5)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:18:11.998926 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695e4afb-acfe-4cae-b14a-5576ce060b8c admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:18:12.138399 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-695e4afb-acfe-4cae-b14a-5576ce060b8c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a406f08c603e4bdcb7596d6b1d59a8e5', 'request_id': 'req-695e4afb-acfe-4cae-b14a-5576ce060b8c'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:18:12.144693 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-695e4afb-acfe-4cae-b14a-5576ce060b8c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a406f08c603e4bdcb7596d6b1d59a8e5', 'request_id': 'req-695e4afb-acfe-4cae-b14a-5576ce060b8c'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:18:12.151305 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-695e4afb-acfe-4cae-b14a-5576ce060b8c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a406f08c603e4bdcb7596d6b1d59a8e5', 'request_id': 'req-695e4afb-acfe-4cae-b14a-5576ce060b8c'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:18:12.230364 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-695e4afb-acfe-4cae-b14a-5576ce060b8c admin admin] The token cache is being invalidate because project a406f08c603e4bdcb7596d6b1d59a8e5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81676) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 10 12:18:12.230580 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-695e4afb-acfe-4cae-b14a-5576ce060b8c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:18:12.231242 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-695e4afb-acfe-4cae-b14a-5576ce060b8c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:18:12.232805 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1161/2346] 10.222.0.46 () {58 vars in 1233 bytes} [Mon Oct 10 12:18:11 2022] DELETE /identity/v3/projects/a406f08c603e4bdcb7596d6b1d59a8e5 => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:18:12.237784 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e66755e-0db3-46ec-a19f-e14af31965a7 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:18:12.238272 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e66755e-0db3-46ec-a19f-e14af31965a7 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:18:12.238380 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e66755e-0db3-46ec-a19f-e14af31965a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:18:12.257535 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-0e66755e-0db3-46ec-a19f-e14af31965a7 None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:18:12.258787 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-0e66755e-0db3-46ec-a19f-e14af31965a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:18:12.316087 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1186/2347] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:18:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:18:12.323928 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca0576a-f688-421b-966b-d5bef886dedd None None] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:18:12.324220 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca0576a-f688-421b-966b-d5bef886dedd None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:18:12.324503 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca0576a-f688-421b-966b-d5bef886dedd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:18:12.344761 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.auth.core [None req-6ca0576a-f688-421b-966b-d5bef886dedd None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:18:12.345960 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-6ca0576a-f688-421b-966b-d5bef886dedd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:18:12.354682 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1162/2348] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:18:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:18:12.358971 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e66755e-0db3-46ec-a19f-e14af31965a7 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:18:12.368437 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e66755e-0db3-46ec-a19f-e14af31965a7 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:18:12.372934 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35d9b9a-4f7c-4f31-91d2-ef12c6d873f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:18:12.373557 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35d9b9a-4f7c-4f31-91d2-ef12c6d873f4 admin admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:18:12.373834 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35d9b9a-4f7c-4f31-91d2-ef12c6d873f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:18:12.374121 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35d9b9a-4f7c-4f31-91d2-ef12c6d873f4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:18:12.374794 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35d9b9a-4f7c-4f31-91d2-ef12c6d873f4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 12:18:12.375100 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35d9b9a-4f7c-4f31-91d2-ef12c6d873f4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:18:12.376679 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35d9b9a-4f7c-4f31-91d2-ef12c6d873f4 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:18:12.390118 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1187/2349] 10.222.0.46 () {58 vars in 1153 bytes} [Mon Oct 10 12:18:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 12:18:12.394926 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca0576a-f688-421b-966b-d5bef886dedd None None] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:18:12.404202 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca0576a-f688-421b-966b-d5bef886dedd None None] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:18:12.409495 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1e8e2b-84df-4480-8b4e-f12ca8599bba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:18:12.410132 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1e8e2b-84df-4480-8b4e-f12ca8599bba admin admin] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:18:12.410416 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1e8e2b-84df-4480-8b4e-f12ca8599bba admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:18:12.410717 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1e8e2b-84df-4480-8b4e-f12ca8599bba admin admin] PATH_INFO: `/v3/projects` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:18:12.412411 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1e8e2b-84df-4480-8b4e-f12ca8599bba admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:18:12.413312 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1e8e2b-84df-4480-8b4e-f12ca8599bba admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:18:12.489962 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1163/2350] 10.222.0.46 () {60 vars in 1153 bytes} [Mon Oct 10 12:18:12 2022] POST /identity/v3/projects => generated 375 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 12:18:12.494195 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35d9b9a-4f7c-4f31-91d2-ef12c6d873f4 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:18:12.504666 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35d9b9a-4f7c-4f31-91d2-ef12c6d873f4 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:18:12.562918 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853236f1-3678-4515-a83a-d8238c2937dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:18:12.563524 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853236f1-3678-4515-a83a-d8238c2937dd admin admin] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:18:12.563807 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853236f1-3678-4515-a83a-d8238c2937dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:18:12.564082 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853236f1-3678-4515-a83a-d8238c2937dd admin admin] PATH_INFO: `/v3/users` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:18:12.564740 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853236f1-3678-4515-a83a-d8238c2937dd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:18:12.567304 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853236f1-3678-4515-a83a-d8238c2937dd admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:18:12.683154 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1188/2351] 10.222.0.46 () {60 vars in 1144 bytes} [Mon Oct 10 12:18:12 2022] POST /identity/v3/users => generated 288 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 10 12:18:12.687987 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1e8e2b-84df-4480-8b4e-f12ca8599bba admin admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:18:12.698976 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1e8e2b-84df-4480-8b4e-f12ca8599bba admin admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:18:12.703752 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a5dc88-62cc-4ec0-81d2-5b0e21f78400 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:18:12.704349 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a5dc88-62cc-4ec0-81d2-5b0e21f78400 admin admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:18:12.704630 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a5dc88-62cc-4ec0-81d2-5b0e21f78400 admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:18:12.704906 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a5dc88-62cc-4ec0-81d2-5b0e21f78400 admin admin] PATH_INFO: `/v3/roles` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:18:12.705549 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a5dc88-62cc-4ec0-81d2-5b0e21f78400 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:18:12.707242 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a5dc88-62cc-4ec0-81d2-5b0e21f78400 admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:18:12.717565 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1164/2352] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 10 12:18:12 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 12:18:12.721786 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853236f1-3678-4515-a83a-d8238c2937dd admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:18:12.732119 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853236f1-3678-4515-a83a-d8238c2937dd admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:18:12.737071 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f77086-ec2d-4377-b508-5314d7cc9b88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:18:12.737632 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f77086-ec2d-4377-b508-5314d7cc9b88 admin admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:18:12.737958 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f77086-ec2d-4377-b508-5314d7cc9b88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:18:12.738231 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f77086-ec2d-4377-b508-5314d7cc9b88 admin admin] PATH_INFO: `/v3/roles` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:18:12.739447 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f77086-ec2d-4377-b508-5314d7cc9b88 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:18:12.741028 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f77086-ec2d-4377-b508-5314d7cc9b88 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:18:12.751633 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1189/2353] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 10 12:18:12 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 10 12:18:12.756519 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a5dc88-62cc-4ec0-81d2-5b0e21f78400 admin admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:18:12.766380 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a5dc88-62cc-4ec0-81d2-5b0e21f78400 admin admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:18:12.771190 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fb3469-db38-4fed-a4c9-30a49d34709e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:18:12.771836 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fb3469-db38-4fed-a4c9-30a49d34709e admin admin] REQUEST_METHOD: `PUT` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:18:12.772115 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fb3469-db38-4fed-a4c9-30a49d34709e admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:18:12.772388 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fb3469-db38-4fed-a4c9-30a49d34709e admin admin] PATH_INFO: `/v3/projects/dd4516dd621242208431a2cfec87d0fd/users/4f8ccc8ce879462db9ca6e0b9ee79f55/roles/b2cb2a6e818e47338ce071e6fedfc673` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:18:12.787626 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01fb3469-db38-4fed-a4c9-30a49d34709e admin admin] RBAC: Authorizing `identity:create_grant(project_id=dd4516dd621242208431a2cfec87d0fd, user_id=4f8ccc8ce879462db9ca6e0b9ee79f55, role_id=b2cb2a6e818e47338ce071e6fedfc673)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:18:12.790460 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01fb3469-db38-4fed-a4c9-30a49d34709e admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:18:12.866234 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1165/2354] 10.222.0.46 () {60 vars in 1483 bytes} [Mon Oct 10 12:18:12 2022] PUT /identity/v3/projects/dd4516dd621242208431a2cfec87d0fd/users/4f8ccc8ce879462db9ca6e0b9ee79f55/roles/b2cb2a6e818e47338ce071e6fedfc673 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:18:12.876455 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66108dc1-7356-48a8-b2b9-10e799f29e4e None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:18:12.876781 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66108dc1-7356-48a8-b2b9-10e799f29e4e None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:18:12.877053 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66108dc1-7356-48a8-b2b9-10e799f29e4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:18:12.904868 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-66108dc1-7356-48a8-b2b9-10e799f29e4e None None] MFA Rules not processed for user `4f8ccc8ce879462db9ca6e0b9ee79f55`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:18:12.906049 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-66108dc1-7356-48a8-b2b9-10e799f29e4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:18:12.947667 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:18:12.970578 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1190/2355] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:18:12 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:18:12.975691 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fb3469-db38-4fed-a4c9-30a49d34709e admin admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:18:13.023265 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:18:13.040614 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fb3469-db38-4fed-a4c9-30a49d34709e admin admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:18:13.045422 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b3376f-3f25-4c27-8c0d-09f3a80e7118 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:18:13.050126 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b3376f-3f25-4c27-8c0d-09f3a80e7118 admin admin] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:18:13.050126 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b3376f-3f25-4c27-8c0d-09f3a80e7118 admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:18:13.050126 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b3376f-3f25-4c27-8c0d-09f3a80e7118 admin admin] PATH_INFO: `/v3/projects` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:18:13.050126 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b3376f-3f25-4c27-8c0d-09f3a80e7118 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:18:13.050126 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b3376f-3f25-4c27-8c0d-09f3a80e7118 admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:18:13.146298 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1166/2356] 10.222.0.46 () {60 vars in 1153 bytes} [Mon Oct 10 12:18:12 2022] POST /identity/v3/projects => generated 375 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 10 12:18:13.150901 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66108dc1-7356-48a8-b2b9-10e799f29e4e None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:18:13.162781 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66108dc1-7356-48a8-b2b9-10e799f29e4e None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:18:13.219069 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93fb7c4-c38b-41a6-a701-e1c292e65935 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:18:13.219696 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93fb7c4-c38b-41a6-a701-e1c292e65935 admin admin] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:18:13.219972 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93fb7c4-c38b-41a6-a701-e1c292e65935 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:18:13.220238 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93fb7c4-c38b-41a6-a701-e1c292e65935 admin admin] PATH_INFO: `/v3/users` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:18:13.220859 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93fb7c4-c38b-41a6-a701-e1c292e65935 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:18:13.222778 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93fb7c4-c38b-41a6-a701-e1c292e65935 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:18:13.317134 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1191/2357] 10.222.0.46 () {60 vars in 1144 bytes} [Mon Oct 10 12:18:13 2022] POST /identity/v3/users => generated 294 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 12:18:13.321133 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b3376f-3f25-4c27-8c0d-09f3a80e7118 admin admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:18:13.331402 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b3376f-3f25-4c27-8c0d-09f3a80e7118 admin admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:18:13.336379 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393629b7-80c3-4a55-82ec-5c2aa5fbd500 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:18:13.337014 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393629b7-80c3-4a55-82ec-5c2aa5fbd500 admin admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:18:13.337298 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393629b7-80c3-4a55-82ec-5c2aa5fbd500 admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:18:13.337577 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393629b7-80c3-4a55-82ec-5c2aa5fbd500 admin admin] PATH_INFO: `/v3/roles` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:18:13.338264 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393629b7-80c3-4a55-82ec-5c2aa5fbd500 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:18:13.339910 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393629b7-80c3-4a55-82ec-5c2aa5fbd500 admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:18:13.351004 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1167/2358] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 10 12:18:13 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 12:18:13.355247 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93fb7c4-c38b-41a6-a701-e1c292e65935 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:18:13.366242 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93fb7c4-c38b-41a6-a701-e1c292e65935 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:18:13.370930 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5198046-0b93-40c7-9515-d1ec3d586beb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:18:13.371581 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5198046-0b93-40c7-9515-d1ec3d586beb admin admin] REQUEST_METHOD: `PUT` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:18:13.371863 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5198046-0b93-40c7-9515-d1ec3d586beb admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:18:13.372162 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5198046-0b93-40c7-9515-d1ec3d586beb admin admin] PATH_INFO: `/v3/projects/f135cbda131a4684b87343a7f96a4495/users/707d245ce8f54577a29ff5faf2071721/roles/0d822a0674b6495fac623877a3b38025` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:18:13.386961 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5198046-0b93-40c7-9515-d1ec3d586beb admin admin] RBAC: Authorizing `identity:create_grant(project_id=f135cbda131a4684b87343a7f96a4495, user_id=707d245ce8f54577a29ff5faf2071721, role_id=0d822a0674b6495fac623877a3b38025)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:18:13.389603 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5198046-0b93-40c7-9515-d1ec3d586beb admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:18:13.500036 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1192/2359] 10.222.0.46 () {60 vars in 1483 bytes} [Mon Oct 10 12:18:13 2022] PUT /identity/v3/projects/f135cbda131a4684b87343a7f96a4495/users/707d245ce8f54577a29ff5faf2071721/roles/0d822a0674b6495fac623877a3b38025 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:18:13.504102 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393629b7-80c3-4a55-82ec-5c2aa5fbd500 admin admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:18:13.514289 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393629b7-80c3-4a55-82ec-5c2aa5fbd500 admin admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:18:13.571058 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b4e6c6-206a-490e-848f-dedc3cd99966 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:18:13.571588 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b4e6c6-206a-490e-848f-dedc3cd99966 admin admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:18:13.571872 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b4e6c6-206a-490e-848f-dedc3cd99966 admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:18:13.572223 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b4e6c6-206a-490e-848f-dedc3cd99966 admin admin] PATH_INFO: `/v3/roles` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:18:13.572878 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b4e6c6-206a-490e-848f-dedc3cd99966 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:18:13.574463 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b4e6c6-206a-490e-848f-dedc3cd99966 admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:18:13.584918 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1168/2360] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 10 12:18:13 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 12:18:13.590398 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5198046-0b93-40c7-9515-d1ec3d586beb admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:18:13.600346 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5198046-0b93-40c7-9515-d1ec3d586beb admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:18:13.605015 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9ded7a-b69f-4c77-8f8e-857b6da37221 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:18:13.605665 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9ded7a-b69f-4c77-8f8e-857b6da37221 admin admin] REQUEST_METHOD: `PUT` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:18:13.605947 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9ded7a-b69f-4c77-8f8e-857b6da37221 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:18:13.606229 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9ded7a-b69f-4c77-8f8e-857b6da37221 admin admin] PATH_INFO: `/v3/projects/f135cbda131a4684b87343a7f96a4495/users/707d245ce8f54577a29ff5faf2071721/roles/0d822a0674b6495fac623877a3b38025` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:18:13.608165 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9ded7a-b69f-4c77-8f8e-857b6da37221 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f135cbda131a4684b87343a7f96a4495, user_id=707d245ce8f54577a29ff5faf2071721, role_id=0d822a0674b6495fac623877a3b38025)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:18:13.610819 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9ded7a-b69f-4c77-8f8e-857b6da37221 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:18:13.622805 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1193/2361] 10.222.0.46 () {60 vars in 1483 bytes} [Mon Oct 10 12:18:13 2022] PUT /identity/v3/projects/f135cbda131a4684b87343a7f96a4495/users/707d245ce8f54577a29ff5faf2071721/roles/0d822a0674b6495fac623877a3b38025 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:18:13.634311 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fc0ab7-d90a-48da-b6ed-1a95b0072f4f None None] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:18:13.636419 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fc0ab7-d90a-48da-b6ed-1a95b0072f4f None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:18:13.636419 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fc0ab7-d90a-48da-b6ed-1a95b0072f4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:18:13.663132 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.auth.core [None req-f2fc0ab7-d90a-48da-b6ed-1a95b0072f4f None None] MFA Rules not processed for user `707d245ce8f54577a29ff5faf2071721`. Rule list: `[]` (Enabled: `True`). {{(pid=81676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:18:13.664328 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-f2fc0ab7-d90a-48da-b6ed-1a95b0072f4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:18:13.731166 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1169/2362] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:18:13 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:18:13.742311 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9ded7a-b69f-4c77-8f8e-857b6da37221 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:18:13.744000 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-6b9ded7a-b69f-4c77-8f8e-857b6da37221 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:18:13.801723 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9ded7a-b69f-4c77-8f8e-857b6da37221 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:18:13.805382 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd26d012-f67b-48b3-9190-5d6bb87016ed service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3ae093470484b93b23dcfae1657df9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad3772f7b6a47c28b7ba37a9316a3c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:18:13.805987 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd26d012-f67b-48b3-9190-5d6bb87016ed service cinder] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:18:13.806270 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd26d012-f67b-48b3-9190-5d6bb87016ed service cinder] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:18:13.806545 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd26d012-f67b-48b3-9190-5d6bb87016ed service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:18:13.853588 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:18:13.863206 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd26d012-f67b-48b3-9190-5d6bb87016ed service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:18:13.864626 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd26d012-f67b-48b3-9190-5d6bb87016ed service cinder] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:18:13.873413 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1194/2363] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 10 12:18:13 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:18:19.013617 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2fc0ab7-d90a-48da-b6ed-1a95b0072f4f None None] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:18:19.015855 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-f2fc0ab7-d90a-48da-b6ed-1a95b0072f4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:18:19.060966 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:18:19.092529 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2fc0ab7-d90a-48da-b6ed-1a95b0072f4f None None] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:18:19.098350 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d2b99f-24f2-4d6b-a15b-e8f87c957d45 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f130bbb69e546ce99b4e14de81d7e54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad3772f7b6a47c28b7ba37a9316a3c1', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:18:19.098994 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d2b99f-24f2-4d6b-a15b-e8f87c957d45 service nova] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:18:19.099283 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d2b99f-24f2-4d6b-a15b-e8f87c957d45 service nova] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:18:19.099556 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d2b99f-24f2-4d6b-a15b-e8f87c957d45 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:18:19.101372 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-91d2b99f-24f2-4d6b-a15b-e8f87c957d45 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:18:19.185795 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d2b99f-24f2-4d6b-a15b-e8f87c957d45 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:18:19.187513 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d2b99f-24f2-4d6b-a15b-e8f87c957d45 service nova] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:18:19.206837 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1170/2364] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 10 12:18:19 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:18:22.308846 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd26d012-f67b-48b3-9190-5d6bb87016ed service cinder] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:18:22.310250 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-fd26d012-f67b-48b3-9190-5d6bb87016ed service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:18:22.321370 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd26d012-f67b-48b3-9190-5d6bb87016ed service cinder] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:18:22.325419 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f91feb-0e96-4032-981e-cfd43e30cf6d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3ae093470484b93b23dcfae1657df9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad3772f7b6a47c28b7ba37a9316a3c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:18:22.326016 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f91feb-0e96-4032-981e-cfd43e30cf6d service cinder] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:18:22.326293 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f91feb-0e96-4032-981e-cfd43e30cf6d service cinder] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:18:22.326595 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f91feb-0e96-4032-981e-cfd43e30cf6d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:18:22.335556 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f91feb-0e96-4032-981e-cfd43e30cf6d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:18:22.336956 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f91feb-0e96-4032-981e-cfd43e30cf6d service cinder] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:18:22.347261 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1195/2365] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 10 12:18:22 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:18:22.965778 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d2b99f-24f2-4d6b-a15b-e8f87c957d45 service nova] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:18:22.975599 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d2b99f-24f2-4d6b-a15b-e8f87c957d45 service nova] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:18:22.981420 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb3d5a9-634d-4f94-bc02-1ed9580cea1a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3ae093470484b93b23dcfae1657df9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad3772f7b6a47c28b7ba37a9316a3c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:18:22.981420 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb3d5a9-634d-4f94-bc02-1ed9580cea1a service cinder] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:18:22.981420 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb3d5a9-634d-4f94-bc02-1ed9580cea1a service cinder] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:18:22.981420 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb3d5a9-634d-4f94-bc02-1ed9580cea1a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:18:22.982445 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-deb3d5a9-634d-4f94-bc02-1ed9580cea1a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:18:22.996501 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb3d5a9-634d-4f94-bc02-1ed9580cea1a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:18:22.997924 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb3d5a9-634d-4f94-bc02-1ed9580cea1a service cinder] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:18:23.013633 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1171/2366] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 10 12:18:22 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:18:26.725635 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f91feb-0e96-4032-981e-cfd43e30cf6d service cinder] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:18:26.739263 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f91feb-0e96-4032-981e-cfd43e30cf6d service cinder] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:18:26.744795 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdebce8a-9137-4fd2-ad23-34b1baae7783 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f130bbb69e546ce99b4e14de81d7e54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad3772f7b6a47c28b7ba37a9316a3c1', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:18:26.746057 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdebce8a-9137-4fd2-ad23-34b1baae7783 service nova] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:18:26.746345 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdebce8a-9137-4fd2-ad23-34b1baae7783 service nova] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:18:26.746616 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdebce8a-9137-4fd2-ad23-34b1baae7783 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:18:26.748489 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-cdebce8a-9137-4fd2-ad23-34b1baae7783 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:18:26.839811 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdebce8a-9137-4fd2-ad23-34b1baae7783 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:18:26.841315 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdebce8a-9137-4fd2-ad23-34b1baae7783 service nova] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:18:26.861064 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1196/2367] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 10 12:18:26 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 10 12:18:30.663709 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb3d5a9-634d-4f94-bc02-1ed9580cea1a service cinder] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:18:30.665167 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-deb3d5a9-634d-4f94-bc02-1ed9580cea1a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:18:30.723927 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb3d5a9-634d-4f94-bc02-1ed9580cea1a service cinder] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:18:30.727649 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f0c8e9-9bef-4c68-8355-993efd0b33b6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58b1c7c85302421cba823ace4201861f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad3772f7b6a47c28b7ba37a9316a3c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:18:30.728256 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f0c8e9-9bef-4c68-8355-993efd0b33b6 service neutron] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:18:30.728542 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f0c8e9-9bef-4c68-8355-993efd0b33b6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:18:30.728824 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f0c8e9-9bef-4c68-8355-993efd0b33b6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:18:30.730628 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-f1f0c8e9-9bef-4c68-8355-993efd0b33b6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:18:30.744563 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f0c8e9-9bef-4c68-8355-993efd0b33b6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:18:30.746005 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f0c8e9-9bef-4c68-8355-993efd0b33b6 service neutron] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:18:30.758768 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1172/2368] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 10 12:18:30 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:18:36.858235 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=52 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:18:40.754039 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=84 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:18:55.727449 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdebce8a-9137-4fd2-ad23-34b1baae7783 service nova] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:18:55.729782 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-cdebce8a-9137-4fd2-ad23-34b1baae7783 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:18:55.749648 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdebce8a-9137-4fd2-ad23-34b1baae7783 service nova] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:18:55.755235 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839e17c6-96fa-493d-937d-e0165afffd29 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58b1c7c85302421cba823ace4201861f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad3772f7b6a47c28b7ba37a9316a3c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:18:55.758969 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839e17c6-96fa-493d-937d-e0165afffd29 service neutron] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:18:55.759774 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839e17c6-96fa-493d-937d-e0165afffd29 service neutron] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:18:55.760472 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839e17c6-96fa-493d-937d-e0165afffd29 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:18:55.766192 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-839e17c6-96fa-493d-937d-e0165afffd29 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:18:55.782980 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839e17c6-96fa-493d-937d-e0165afffd29 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:18:55.785696 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839e17c6-96fa-493d-937d-e0165afffd29 service neutron] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:18:55.802232 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1197/2369] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 10 12:18:55 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 76 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:18:55.828342 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f0c8e9-9bef-4c68-8355-993efd0b33b6 service neutron] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:18:55.829873 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-f1f0c8e9-9bef-4c68-8355-993efd0b33b6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:18:55.848209 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f0c8e9-9bef-4c68-8355-993efd0b33b6 service neutron] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:18:55.855829 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c4deac-6445-4118-a5cc-651c96aa0bea service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f130bbb69e546ce99b4e14de81d7e54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad3772f7b6a47c28b7ba37a9316a3c1', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:18:55.856720 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c4deac-6445-4118-a5cc-651c96aa0bea service nova] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:18:55.857143 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c4deac-6445-4118-a5cc-651c96aa0bea service nova] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:18:55.857569 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c4deac-6445-4118-a5cc-651c96aa0bea service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:18:55.862243 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-82c4deac-6445-4118-a5cc-651c96aa0bea service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:18:55.880980 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c4deac-6445-4118-a5cc-651c96aa0bea service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:18:55.882964 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c4deac-6445-4118-a5cc-651c96aa0bea service nova] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:18:55.904097 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1173/2370] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 10 12:18:55 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 76 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:18:56.657550 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b50212-17e2-479d-bba4-409cf0ecacc9 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:18:56.657994 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b50212-17e2-479d-bba4-409cf0ecacc9 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:18:56.658453 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b50212-17e2-479d-bba4-409cf0ecacc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:18:56.703405 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-30b50212-17e2-479d-bba4-409cf0ecacc9 None None] MFA Rules not processed for user `fef422264d0c4a43a0aa1b9eff2251bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:18:56.705346 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-30b50212-17e2-479d-bba4-409cf0ecacc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:18:56.718687 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1198/2371] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:18:56 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:18:57.069577 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c4deac-6445-4118-a5cc-651c96aa0bea service nova] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:18:57.079928 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c4deac-6445-4118-a5cc-651c96aa0bea service nova] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:18:57.083828 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3019b1fa-c341-42a4-8880-7aaa1463157f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58b1c7c85302421cba823ace4201861f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad3772f7b6a47c28b7ba37a9316a3c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:18:57.084492 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3019b1fa-c341-42a4-8880-7aaa1463157f service neutron] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:18:57.084777 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3019b1fa-c341-42a4-8880-7aaa1463157f service neutron] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:18:57.085055 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3019b1fa-c341-42a4-8880-7aaa1463157f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:18:57.087020 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-3019b1fa-c341-42a4-8880-7aaa1463157f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:18:57.098555 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3019b1fa-c341-42a4-8880-7aaa1463157f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:18:57.100043 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3019b1fa-c341-42a4-8880-7aaa1463157f service neutron] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:18:57.111057 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1174/2372] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 10 12:18:57 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:18:57.508708 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b50212-17e2-479d-bba4-409cf0ecacc9 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:18:57.586468 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b50212-17e2-479d-bba4-409cf0ecacc9 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:18:57.593130 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479584a1-a504-4e4f-af3c-3e490483f9bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:18:57.594139 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479584a1-a504-4e4f-af3c-3e490483f9bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:18:57.594618 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479584a1-a504-4e4f-af3c-3e490483f9bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:18:57.595167 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479584a1-a504-4e4f-af3c-3e490483f9bd admin admin] PATH_INFO: `/v3/users/4f8ccc8ce879462db9ca6e0b9ee79f55` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:18:57.597170 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479584a1-a504-4e4f-af3c-3e490483f9bd admin admin] RBAC: Authorizing `identity:delete_user(user_id=4f8ccc8ce879462db9ca6e0b9ee79f55)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:18:57.600688 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479584a1-a504-4e4f-af3c-3e490483f9bd admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:18:57.628829 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3019b1fa-c341-42a4-8880-7aaa1463157f service neutron] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:18:57.630703 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-3019b1fa-c341-42a4-8880-7aaa1463157f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:18:57.714293 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3019b1fa-c341-42a4-8880-7aaa1463157f service neutron] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:18:57.721464 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665827a3-76cf-4b1a-9813-b17c48b58d2a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e8f9bb1776d47169ef7febb57c954cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad3772f7b6a47c28b7ba37a9316a3c1', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:18:57.722167 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665827a3-76cf-4b1a-9813-b17c48b58d2a service placement] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:18:57.722508 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665827a3-76cf-4b1a-9813-b17c48b58d2a service placement] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:18:57.722897 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665827a3-76cf-4b1a-9813-b17c48b58d2a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:18:57.725040 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-665827a3-76cf-4b1a-9813-b17c48b58d2a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:18:57.739764 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665827a3-76cf-4b1a-9813-b17c48b58d2a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:18:57.741589 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665827a3-76cf-4b1a-9813-b17c48b58d2a service placement] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:18:57.753194 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1175/2373] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 10 12:18:57 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 10 12:18:57.836382 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-479584a1-a504-4e4f-af3c-3e490483f9bd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4f8ccc8ce879462db9ca6e0b9ee79f55'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:18:57.846086 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-479584a1-a504-4e4f-af3c-3e490483f9bd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4f8ccc8ce879462db9ca6e0b9ee79f55'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:18:57.846867 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-479584a1-a504-4e4f-af3c-3e490483f9bd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4f8ccc8ce879462db9ca6e0b9ee79f55'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:18:57.914291 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-479584a1-a504-4e4f-af3c-3e490483f9bd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4f8ccc8ce879462db9ca6e0b9ee79f55'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:18:57.928711 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-479584a1-a504-4e4f-af3c-3e490483f9bd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4f8ccc8ce879462db9ca6e0b9ee79f55'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:18:57.935780 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1199/2374] 10.222.0.46 () {58 vars in 1224 bytes} [Mon Oct 10 12:18:57 2022] DELETE /identity/v3/users/4f8ccc8ce879462db9ca6e0b9ee79f55 => generated 0 bytes in 428 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:18:57.940435 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9af5ae-16dd-4212-a3a3-993dbd1d144c None None] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:18:57.940744 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9af5ae-16dd-4212-a3a3-993dbd1d144c None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:18:57.941024 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9af5ae-16dd-4212-a3a3-993dbd1d144c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:18:57.960086 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.auth.core [None req-5f9af5ae-16dd-4212-a3a3-993dbd1d144c None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:18:57.961312 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-5f9af5ae-16dd-4212-a3a3-993dbd1d144c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:18:58.025642 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1176/2375] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:18:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:18:58.037779 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479584a1-a504-4e4f-af3c-3e490483f9bd admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:18:58.039249 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-479584a1-a504-4e4f-af3c-3e490483f9bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:18:58.122554 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479584a1-a504-4e4f-af3c-3e490483f9bd admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:18:58.126344 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d32da3e-93c4-4791-9f84-d7136289ba88 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58b1c7c85302421cba823ace4201861f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad3772f7b6a47c28b7ba37a9316a3c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:18:58.127004 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d32da3e-93c4-4791-9f84-d7136289ba88 service neutron] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:18:58.127295 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d32da3e-93c4-4791-9f84-d7136289ba88 service neutron] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:18:58.127562 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d32da3e-93c4-4791-9f84-d7136289ba88 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:18:58.135241 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:18:58.138238 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d32da3e-93c4-4791-9f84-d7136289ba88 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:18:58.140030 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d32da3e-93c4-4791-9f84-d7136289ba88 service neutron] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:18:58.151227 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1200/2376] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 10 12:18:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:18:58.798333 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9af5ae-16dd-4212-a3a3-993dbd1d144c None None] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:18:58.799940 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-5f9af5ae-16dd-4212-a3a3-993dbd1d144c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:18:58.813672 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9af5ae-16dd-4212-a3a3-993dbd1d144c None None] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:18:58.819258 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bef26d1-9cee-49fe-ac5a-f4bb1d9f721b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:18:58.820370 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bef26d1-9cee-49fe-ac5a-f4bb1d9f721b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:18:58.820836 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bef26d1-9cee-49fe-ac5a-f4bb1d9f721b admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:18:58.821105 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bef26d1-9cee-49fe-ac5a-f4bb1d9f721b admin admin] PATH_INFO: `/v3/projects/dd4516dd621242208431a2cfec87d0fd` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:18:58.822403 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bef26d1-9cee-49fe-ac5a-f4bb1d9f721b admin admin] RBAC: Authorizing `identity:delete_project(project_id=dd4516dd621242208431a2cfec87d0fd)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:18:58.824387 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bef26d1-9cee-49fe-ac5a-f4bb1d9f721b admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:18:59.141662 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-1bef26d1-9cee-49fe-ac5a-f4bb1d9f721b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dd4516dd621242208431a2cfec87d0fd', 'request_id': 'req-1bef26d1-9cee-49fe-ac5a-f4bb1d9f721b'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:18:59.147546 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-1bef26d1-9cee-49fe-ac5a-f4bb1d9f721b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dd4516dd621242208431a2cfec87d0fd', 'request_id': 'req-1bef26d1-9cee-49fe-ac5a-f4bb1d9f721b'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:18:59.148623 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:18:59.155597 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-1bef26d1-9cee-49fe-ac5a-f4bb1d9f721b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dd4516dd621242208431a2cfec87d0fd', 'request_id': 'req-1bef26d1-9cee-49fe-ac5a-f4bb1d9f721b'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:18:59.265490 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-1bef26d1-9cee-49fe-ac5a-f4bb1d9f721b admin admin] The token cache is being invalidate because project dd4516dd621242208431a2cfec87d0fd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81676) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 10 12:18:59.265756 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-1bef26d1-9cee-49fe-ac5a-f4bb1d9f721b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:18:59.266317 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-1bef26d1-9cee-49fe-ac5a-f4bb1d9f721b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:18:59.268012 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1177/2377] 10.222.0.46 () {58 vars in 1233 bytes} [Mon Oct 10 12:18:58 2022] DELETE /identity/v3/projects/dd4516dd621242208431a2cfec87d0fd => generated 0 bytes in 470 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:18:59.272283 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d32da3e-93c4-4791-9f84-d7136289ba88 service neutron] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:18:59.283028 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d32da3e-93c4-4791-9f84-d7136289ba88 service neutron] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:18:59.343535 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3bdf71-120f-45c5-8523-e69bf0b22b4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:18:59.344144 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3bdf71-120f-45c5-8523-e69bf0b22b4a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:18:59.344427 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3bdf71-120f-45c5-8523-e69bf0b22b4a admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:18:59.344703 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3bdf71-120f-45c5-8523-e69bf0b22b4a admin admin] PATH_INFO: `/v3/users/707d245ce8f54577a29ff5faf2071721` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:18:59.345864 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3bdf71-120f-45c5-8523-e69bf0b22b4a admin admin] RBAC: Authorizing `identity:delete_user(user_id=707d245ce8f54577a29ff5faf2071721)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:18:59.347875 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3bdf71-120f-45c5-8523-e69bf0b22b4a admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:18:59.631649 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-df3bdf71-120f-45c5-8523-e69bf0b22b4a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '707d245ce8f54577a29ff5faf2071721'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:18:59.639859 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-df3bdf71-120f-45c5-8523-e69bf0b22b4a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '707d245ce8f54577a29ff5faf2071721'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:18:59.640699 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-df3bdf71-120f-45c5-8523-e69bf0b22b4a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '707d245ce8f54577a29ff5faf2071721'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:18:59.760861 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-df3bdf71-120f-45c5-8523-e69bf0b22b4a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '707d245ce8f54577a29ff5faf2071721'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:18:59.776011 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-df3bdf71-120f-45c5-8523-e69bf0b22b4a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '707d245ce8f54577a29ff5faf2071721'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:18:59.789467 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1201/2378] 10.222.0.46 () {58 vars in 1224 bytes} [Mon Oct 10 12:18:59 2022] DELETE /identity/v3/users/707d245ce8f54577a29ff5faf2071721 => generated 0 bytes in 517 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 10 12:19:00.203492 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bef26d1-9cee-49fe-ac5a-f4bb1d9f721b admin admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:19:00.204954 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-1bef26d1-9cee-49fe-ac5a-f4bb1d9f721b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:19:00.220475 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bef26d1-9cee-49fe-ac5a-f4bb1d9f721b admin admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:19:00.288745 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d43cda0-9477-4a6f-bf0d-b1aee2d87e03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:19:00.289492 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d43cda0-9477-4a6f-bf0d-b1aee2d87e03 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:19:00.289774 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d43cda0-9477-4a6f-bf0d-b1aee2d87e03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:19:00.290718 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d43cda0-9477-4a6f-bf0d-b1aee2d87e03 admin admin] PATH_INFO: `/v3/projects/f135cbda131a4684b87343a7f96a4495` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:19:00.291908 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d43cda0-9477-4a6f-bf0d-b1aee2d87e03 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f135cbda131a4684b87343a7f96a4495)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:19:00.293825 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d43cda0-9477-4a6f-bf0d-b1aee2d87e03 admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:19:00.494490 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-8d43cda0-9477-4a6f-bf0d-b1aee2d87e03 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f135cbda131a4684b87343a7f96a4495', 'request_id': 'req-8d43cda0-9477-4a6f-bf0d-b1aee2d87e03'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:19:00.505292 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-8d43cda0-9477-4a6f-bf0d-b1aee2d87e03 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f135cbda131a4684b87343a7f96a4495', 'request_id': 'req-8d43cda0-9477-4a6f-bf0d-b1aee2d87e03'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:19:00.518487 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-8d43cda0-9477-4a6f-bf0d-b1aee2d87e03 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f135cbda131a4684b87343a7f96a4495', 'request_id': 'req-8d43cda0-9477-4a6f-bf0d-b1aee2d87e03'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:19:00.636571 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-8d43cda0-9477-4a6f-bf0d-b1aee2d87e03 admin admin] The token cache is being invalidate because project f135cbda131a4684b87343a7f96a4495 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81676) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 10 12:19:00.637731 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-8d43cda0-9477-4a6f-bf0d-b1aee2d87e03 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:19:00.637731 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-8d43cda0-9477-4a6f-bf0d-b1aee2d87e03 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:19:00.639817 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1178/2379] 10.222.0.46 () {58 vars in 1233 bytes} [Mon Oct 10 12:19:00 2022] DELETE /identity/v3/projects/f135cbda131a4684b87343a7f96a4495 => generated 0 bytes in 437 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:19:00.645036 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4aa9ad-e3ed-4acc-89b5-bdd1dd5a98c5 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:19:00.645919 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4aa9ad-e3ed-4acc-89b5-bdd1dd5a98c5 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:19:00.645919 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4aa9ad-e3ed-4acc-89b5-bdd1dd5a98c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:19:00.666534 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-9d4aa9ad-e3ed-4acc-89b5-bdd1dd5a98c5 None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:19:00.667854 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-9d4aa9ad-e3ed-4acc-89b5-bdd1dd5a98c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:19:00.745085 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1202/2380] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:19:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 10 12:19:00.745287 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8552a0-8703-4748-a345-a5ab1c48c055 None None] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:19:00.745287 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8552a0-8703-4748-a345-a5ab1c48c055 None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:19:00.745287 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8552a0-8703-4748-a345-a5ab1c48c055 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:19:00.765211 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.auth.core [None req-df8552a0-8703-4748-a345-a5ab1c48c055 None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:19:00.766446 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-df8552a0-8703-4748-a345-a5ab1c48c055 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:19:00.776671 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1179/2381] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:19:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 10 12:19:00.781816 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4aa9ad-e3ed-4acc-89b5-bdd1dd5a98c5 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:19:00.792981 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4aa9ad-e3ed-4acc-89b5-bdd1dd5a98c5 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:19:00.798323 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecec3d0c-2d68-47b8-a11e-fe82b10787cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:19:00.799901 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecec3d0c-2d68-47b8-a11e-fe82b10787cd admin admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:19:00.800196 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecec3d0c-2d68-47b8-a11e-fe82b10787cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:19:00.800485 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecec3d0c-2d68-47b8-a11e-fe82b10787cd admin admin] PATH_INFO: `/v3/domains` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:19:00.801147 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecec3d0c-2d68-47b8-a11e-fe82b10787cd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 12:19:00.801458 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecec3d0c-2d68-47b8-a11e-fe82b10787cd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:19:00.803767 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecec3d0c-2d68-47b8-a11e-fe82b10787cd admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:19:00.818545 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1203/2382] 10.222.0.46 () {58 vars in 1153 bytes} [Mon Oct 10 12:19:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 12:19:00.823371 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8552a0-8703-4748-a345-a5ab1c48c055 None None] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:19:00.834426 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8552a0-8703-4748-a345-a5ab1c48c055 None None] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:19:00.840194 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efa152a-f762-4894-99b4-15586507227e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:19:00.840825 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efa152a-f762-4894-99b4-15586507227e admin admin] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:19:00.841131 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efa152a-f762-4894-99b4-15586507227e admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:19:00.841463 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efa152a-f762-4894-99b4-15586507227e admin admin] PATH_INFO: `/v3/projects` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:19:00.842472 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8efa152a-f762-4894-99b4-15586507227e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:19:00.844283 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8efa152a-f762-4894-99b4-15586507227e admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:19:00.984249 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1180/2383] 10.222.0.46 () {60 vars in 1153 bytes} [Mon Oct 10 12:19:00 2022] POST /identity/v3/projects => generated 385 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 10 12:19:00.988472 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecec3d0c-2d68-47b8-a11e-fe82b10787cd admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:19:00.995698 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:19:01.016337 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecec3d0c-2d68-47b8-a11e-fe82b10787cd admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:19:01.056826 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccc3f21-fedf-4796-98fd-3a46f1cdf901 None None] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:19:01.056826 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccc3f21-fedf-4796-98fd-3a46f1cdf901 None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:19:01.057570 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccc3f21-fedf-4796-98fd-3a46f1cdf901 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:19:01.131541 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a12c6cc-e404-494e-9b1e-30719e84f359 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:19:01.132835 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.auth.core [None req-6ccc3f21-fedf-4796-98fd-3a46f1cdf901 None None] MFA Rules not processed for user `d6430593eecf45d8b891c3b55de4a944`. Rule list: `[]` (Enabled: `True`). {{(pid=81676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:19:01.133546 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a12c6cc-e404-494e-9b1e-30719e84f359 admin admin] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:19:01.135654 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-6ccc3f21-fedf-4796-98fd-3a46f1cdf901 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:19:01.136881 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a12c6cc-e404-494e-9b1e-30719e84f359 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:19:01.137381 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a12c6cc-e404-494e-9b1e-30719e84f359 admin admin] PATH_INFO: `/v3/users` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:19:01.138122 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a12c6cc-e404-494e-9b1e-30719e84f359 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:19:01.141579 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a12c6cc-e404-494e-9b1e-30719e84f359 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:19:01.187793 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=89,keystone:INSERT=3,keystone:DELETE=8 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:19:01.243373 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1181/2384] 10.222.0.46 () {58 vars in 1009 bytes} [Mon Oct 10 12:19:01 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:19:01.256118 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1204/2385] 10.222.0.46 () {60 vars in 1144 bytes} [Mon Oct 10 12:19:00 2022] POST /identity/v3/users => generated 293 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 10 12:19:01.261210 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ccc3f21-fedf-4796-98fd-3a46f1cdf901 None None] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:19:01.274517 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ccc3f21-fedf-4796-98fd-3a46f1cdf901 None None] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:19:01.279476 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57295546-7f63-4470-a239-3bebbcae416d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:19:01.280984 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57295546-7f63-4470-a239-3bebbcae416d admin admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:19:01.280984 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57295546-7f63-4470-a239-3bebbcae416d admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:19:01.281428 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57295546-7f63-4470-a239-3bebbcae416d admin admin] PATH_INFO: `/v3/roles` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:19:01.281919 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57295546-7f63-4470-a239-3bebbcae416d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:19:01.283584 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57295546-7f63-4470-a239-3bebbcae416d admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:19:01.297787 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1182/2386] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 10 12:19:01 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 12:19:01.303744 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a12c6cc-e404-494e-9b1e-30719e84f359 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:19:01.317302 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a12c6cc-e404-494e-9b1e-30719e84f359 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:19:01.323653 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033d7382-b2c0-4b9d-b6e6-c73491d53061 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:19:01.324161 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033d7382-b2c0-4b9d-b6e6-c73491d53061 admin admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:19:01.324435 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033d7382-b2c0-4b9d-b6e6-c73491d53061 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:19:01.324705 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033d7382-b2c0-4b9d-b6e6-c73491d53061 admin admin] PATH_INFO: `/v3/roles` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:19:01.325359 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033d7382-b2c0-4b9d-b6e6-c73491d53061 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:19:01.327591 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033d7382-b2c0-4b9d-b6e6-c73491d53061 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:19:01.346794 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1205/2387] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 10 12:19:01 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 10 12:19:01.349127 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57295546-7f63-4470-a239-3bebbcae416d admin admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:19:01.360284 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57295546-7f63-4470-a239-3bebbcae416d admin admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:19:01.365492 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e45bae2-8cb3-45d1-9913-0531fef52d7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:19:01.366181 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e45bae2-8cb3-45d1-9913-0531fef52d7c admin admin] REQUEST_METHOD: `PUT` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:19:01.367168 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e45bae2-8cb3-45d1-9913-0531fef52d7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:19:01.367168 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e45bae2-8cb3-45d1-9913-0531fef52d7c admin admin] PATH_INFO: `/v3/projects/c1a3a6fae6cf47a181dbec6349a501e5/users/31b4ea75e5834602b758175da0c42502/roles/b2cb2a6e818e47338ce071e6fedfc673` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:19:01.385223 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e45bae2-8cb3-45d1-9913-0531fef52d7c admin admin] RBAC: Authorizing `identity:create_grant(project_id=c1a3a6fae6cf47a181dbec6349a501e5, user_id=31b4ea75e5834602b758175da0c42502, role_id=b2cb2a6e818e47338ce071e6fedfc673)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:19:01.388006 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e45bae2-8cb3-45d1-9913-0531fef52d7c admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:19:01.403491 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033d7382-b2c0-4b9d-b6e6-c73491d53061 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:19:01.407123 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-033d7382-b2c0-4b9d-b6e6-c73491d53061 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:19:01.456050 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1183/2388] 10.222.0.46 () {60 vars in 1483 bytes} [Mon Oct 10 12:19:01 2022] PUT /identity/v3/projects/c1a3a6fae6cf47a181dbec6349a501e5/users/31b4ea75e5834602b758175da0c42502/roles/b2cb2a6e818e47338ce071e6fedfc673 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 10 12:19:01.469744 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cd0577-b554-4659-9c01-84d8c6729a1d None None] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:19:01.470456 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cd0577-b554-4659-9c01-84d8c6729a1d None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:19:01.471039 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cd0577-b554-4659-9c01-84d8c6729a1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:19:01.505435 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.auth.core [None req-f2cd0577-b554-4659-9c01-84d8c6729a1d None None] MFA Rules not processed for user `31b4ea75e5834602b758175da0c42502`. Rule list: `[]` (Enabled: `True`). {{(pid=81676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:19:01.506780 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-f2cd0577-b554-4659-9c01-84d8c6729a1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:19:01.549756 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033d7382-b2c0-4b9d-b6e6-c73491d53061 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:19:01.621658 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1184/2389] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:19:01 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:19:01.624496 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2cd0577-b554-4659-9c01-84d8c6729a1d None None] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:19:01.651914 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502b5803-df6f-4696-b9f4-1523d9f400a3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15ef3d53ea484237adb0b3d64fdc5480', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad3772f7b6a47c28b7ba37a9316a3c1', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:19:01.656217 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502b5803-df6f-4696-b9f4-1523d9f400a3 service swift] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:19:01.656217 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502b5803-df6f-4696-b9f4-1523d9f400a3 service swift] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:19:01.656217 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502b5803-df6f-4696-b9f4-1523d9f400a3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:19:01.659459 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:19:01.717108 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2cd0577-b554-4659-9c01-84d8c6729a1d None None] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:19:01.722798 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12721681-77ca-4e99-92f5-8b666a700625 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:19:01.723488 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12721681-77ca-4e99-92f5-8b666a700625 admin admin] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:19:01.723773 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12721681-77ca-4e99-92f5-8b666a700625 admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:19:01.724051 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12721681-77ca-4e99-92f5-8b666a700625 admin admin] PATH_INFO: `/v3/projects` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:19:01.724926 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12721681-77ca-4e99-92f5-8b666a700625 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:19:01.727473 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12721681-77ca-4e99-92f5-8b666a700625 admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:19:01.729664 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502b5803-df6f-4696-b9f4-1523d9f400a3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:19:01.731497 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502b5803-df6f-4696-b9f4-1523d9f400a3 service swift] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:19:01.741497 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1206/2390] 10.222.0.46 () {60 vars in 1384 bytes} [Mon Oct 10 12:19:01 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 10 12:19:01.810259 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:19:01.818952 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1185/2391] 10.222.0.46 () {60 vars in 1153 bytes} [Mon Oct 10 12:19:01 2022] POST /identity/v3/projects => generated 387 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 10 12:19:01.823240 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502b5803-df6f-4696-b9f4-1523d9f400a3 service swift] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:19:01.837483 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502b5803-df6f-4696-b9f4-1523d9f400a3 service swift] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:19:01.955718 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc96138-6ead-4727-9186-15e5b4a5beda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:19:01.957018 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc96138-6ead-4727-9186-15e5b4a5beda admin admin] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:19:01.957730 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc96138-6ead-4727-9186-15e5b4a5beda admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:19:01.958308 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc96138-6ead-4727-9186-15e5b4a5beda admin admin] PATH_INFO: `/v3/users` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:19:01.959828 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc96138-6ead-4727-9186-15e5b4a5beda admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:19:01.963019 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc96138-6ead-4727-9186-15e5b4a5beda admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:19:02.066262 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1207/2392] 10.222.0.46 () {60 vars in 1144 bytes} [Mon Oct 10 12:19:01 2022] POST /identity/v3/users => generated 300 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 12:19:02.070889 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12721681-77ca-4e99-92f5-8b666a700625 admin admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:19:02.087011 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12721681-77ca-4e99-92f5-8b666a700625 admin admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:19:02.094068 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4b9855-34f5-4861-a239-d832db679c99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:19:02.095740 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4b9855-34f5-4861-a239-d832db679c99 admin admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:19:02.096150 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4b9855-34f5-4861-a239-d832db679c99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:19:02.096726 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4b9855-34f5-4861-a239-d832db679c99 admin admin] PATH_INFO: `/v3/roles` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:19:02.097755 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4b9855-34f5-4861-a239-d832db679c99 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:19:02.100304 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4b9855-34f5-4861-a239-d832db679c99 admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:19:02.117036 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1186/2393] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 10 12:19:02 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 12:19:02.121101 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc96138-6ead-4727-9186-15e5b4a5beda admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:19:02.132527 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc96138-6ead-4727-9186-15e5b4a5beda admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:19:02.139659 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6eae56f-d5f3-4408-b7fe-1e54009a3741 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:19:02.140892 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6eae56f-d5f3-4408-b7fe-1e54009a3741 admin admin] REQUEST_METHOD: `PUT` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:19:02.141315 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6eae56f-d5f3-4408-b7fe-1e54009a3741 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:19:02.141807 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6eae56f-d5f3-4408-b7fe-1e54009a3741 admin admin] PATH_INFO: `/v3/projects/926987f9991e44d387b60d468e63c6ca/users/ad60e58aad0c404198b2a7b0c4845b82/roles/0d822a0674b6495fac623877a3b38025` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:19:02.180620 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6eae56f-d5f3-4408-b7fe-1e54009a3741 admin admin] RBAC: Authorizing `identity:create_grant(project_id=926987f9991e44d387b60d468e63c6ca, user_id=ad60e58aad0c404198b2a7b0c4845b82, role_id=0d822a0674b6495fac623877a3b38025)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:19:02.188402 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6eae56f-d5f3-4408-b7fe-1e54009a3741 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:19:02.273436 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1208/2394] 10.222.0.46 () {60 vars in 1483 bytes} [Mon Oct 10 12:19:02 2022] PUT /identity/v3/projects/926987f9991e44d387b60d468e63c6ca/users/ad60e58aad0c404198b2a7b0c4845b82/roles/0d822a0674b6495fac623877a3b38025 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:19:02.278143 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4b9855-34f5-4861-a239-d832db679c99 admin admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:19:02.304678 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4b9855-34f5-4861-a239-d832db679c99 admin admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:19:02.397609 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b11051a-5132-4285-bb80-c7dd0575f049 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:19:02.398392 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b11051a-5132-4285-bb80-c7dd0575f049 admin admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:19:02.398809 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b11051a-5132-4285-bb80-c7dd0575f049 admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:19:02.399208 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b11051a-5132-4285-bb80-c7dd0575f049 admin admin] PATH_INFO: `/v3/roles` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:19:02.399987 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b11051a-5132-4285-bb80-c7dd0575f049 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:19:02.401974 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b11051a-5132-4285-bb80-c7dd0575f049 admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:19:02.421052 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6eae56f-d5f3-4408-b7fe-1e54009a3741 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:19:02.431573 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1187/2395] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 10 12:19:02 2022] GET /identity/v3/roles => generated 1420 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 10 12:19:02.434413 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6eae56f-d5f3-4408-b7fe-1e54009a3741 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:19:02.440389 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747691fe-04aa-4311-88cb-ba2bc156733c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:19:02.441435 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747691fe-04aa-4311-88cb-ba2bc156733c admin admin] REQUEST_METHOD: `PUT` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:19:02.441891 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747691fe-04aa-4311-88cb-ba2bc156733c admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:19:02.442275 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747691fe-04aa-4311-88cb-ba2bc156733c admin admin] PATH_INFO: `/v3/projects/926987f9991e44d387b60d468e63c6ca/users/ad60e58aad0c404198b2a7b0c4845b82/roles/0d822a0674b6495fac623877a3b38025` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:19:02.445931 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747691fe-04aa-4311-88cb-ba2bc156733c admin admin] RBAC: Authorizing `identity:create_grant(project_id=926987f9991e44d387b60d468e63c6ca, user_id=ad60e58aad0c404198b2a7b0c4845b82, role_id=0d822a0674b6495fac623877a3b38025)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:19:02.451630 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747691fe-04aa-4311-88cb-ba2bc156733c admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:19:02.468938 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1209/2396] 10.222.0.46 () {60 vars in 1483 bytes} [Mon Oct 10 12:19:02 2022] PUT /identity/v3/projects/926987f9991e44d387b60d468e63c6ca/users/ad60e58aad0c404198b2a7b0c4845b82/roles/0d822a0674b6495fac623877a3b38025 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:19:02.481071 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2aeae1-1198-474e-bb9d-c04a686d32f1 None None] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:19:02.481505 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2aeae1-1198-474e-bb9d-c04a686d32f1 None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:19:02.481860 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2aeae1-1198-474e-bb9d-c04a686d32f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:19:02.519228 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.auth.core [None req-8f2aeae1-1198-474e-bb9d-c04a686d32f1 None None] MFA Rules not processed for user `ad60e58aad0c404198b2a7b0c4845b82`. Rule list: `[]` (Enabled: `True`). {{(pid=81676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:19:02.522011 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-8f2aeae1-1198-474e-bb9d-c04a686d32f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:19:02.639161 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1188/2397] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:19:02 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:19:02.653766 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747691fe-04aa-4311-88cb-ba2bc156733c admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:19:02.657302 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-747691fe-04aa-4311-88cb-ba2bc156733c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:19:02.755333 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747691fe-04aa-4311-88cb-ba2bc156733c admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:19:02.759330 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f440436d-913a-4c4e-95d4-53e9a0e657c5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3ae093470484b93b23dcfae1657df9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad3772f7b6a47c28b7ba37a9316a3c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:19:02.760291 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f440436d-913a-4c4e-95d4-53e9a0e657c5 service cinder] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:19:02.760415 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f440436d-913a-4c4e-95d4-53e9a0e657c5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:19:02.760607 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f440436d-913a-4c4e-95d4-53e9a0e657c5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:19:02.774658 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f440436d-913a-4c4e-95d4-53e9a0e657c5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:19:02.777406 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f440436d-913a-4c4e-95d4-53e9a0e657c5 service cinder] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:19:02.789242 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:19:02.799932 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1210/2398] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 10 12:19:02 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:19:04.410017 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2aeae1-1198-474e-bb9d-c04a686d32f1 None None] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:19:04.508206 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2aeae1-1198-474e-bb9d-c04a686d32f1 None None] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:19:04.516117 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3908a532-8085-4e58-81ef-6ebf79669b16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:19:04.516675 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3908a532-8085-4e58-81ef-6ebf79669b16 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:19:04.516907 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3908a532-8085-4e58-81ef-6ebf79669b16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:19:04.517190 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3908a532-8085-4e58-81ef-6ebf79669b16 admin admin] PATH_INFO: `/v3/users/31b4ea75e5834602b758175da0c42502` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:19:04.519325 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3908a532-8085-4e58-81ef-6ebf79669b16 admin admin] RBAC: Authorizing `identity:delete_user(user_id=31b4ea75e5834602b758175da0c42502)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:19:04.521366 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3908a532-8085-4e58-81ef-6ebf79669b16 admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:19:04.537666 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:19:04.771584 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-3908a532-8085-4e58-81ef-6ebf79669b16 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '31b4ea75e5834602b758175da0c42502'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:19:04.782248 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-3908a532-8085-4e58-81ef-6ebf79669b16 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '31b4ea75e5834602b758175da0c42502'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:19:04.783534 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-3908a532-8085-4e58-81ef-6ebf79669b16 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '31b4ea75e5834602b758175da0c42502'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:19:04.882270 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-3908a532-8085-4e58-81ef-6ebf79669b16 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '31b4ea75e5834602b758175da0c42502'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:19:04.905179 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-3908a532-8085-4e58-81ef-6ebf79669b16 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '31b4ea75e5834602b758175da0c42502'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:19:04.912374 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1189/2399] 10.222.0.46 () {58 vars in 1224 bytes} [Mon Oct 10 12:19:04 2022] DELETE /identity/v3/users/31b4ea75e5834602b758175da0c42502 => generated 0 bytes in 503 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:19:04.921294 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21795214-76de-426e-9741-ee4da3dee587 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:19:04.921861 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21795214-76de-426e-9741-ee4da3dee587 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:19:04.921994 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21795214-76de-426e-9741-ee4da3dee587 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:19:04.985736 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-21795214-76de-426e-9741-ee4da3dee587 None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:19:04.987146 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-21795214-76de-426e-9741-ee4da3dee587 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:19:05.080590 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1211/2400] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:19:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:19:05.098629 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3908a532-8085-4e58-81ef-6ebf79669b16 admin admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:19:05.100235 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-3908a532-8085-4e58-81ef-6ebf79669b16 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:19:05.226077 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3908a532-8085-4e58-81ef-6ebf79669b16 admin admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:19:05.230423 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe2dd96-5ef0-4dd6-b755-b6552e5185d2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58b1c7c85302421cba823ace4201861f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad3772f7b6a47c28b7ba37a9316a3c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:19:05.232149 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe2dd96-5ef0-4dd6-b755-b6552e5185d2 service neutron] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:19:05.232149 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe2dd96-5ef0-4dd6-b755-b6552e5185d2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:19:05.232149 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe2dd96-5ef0-4dd6-b755-b6552e5185d2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:19:05.248880 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe2dd96-5ef0-4dd6-b755-b6552e5185d2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:19:05.250365 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe2dd96-5ef0-4dd6-b755-b6552e5185d2 service neutron] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:19:05.282908 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1190/2401] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 10 12:19:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 10 12:19:06.202445 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21795214-76de-426e-9741-ee4da3dee587 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:19:06.205001 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-21795214-76de-426e-9741-ee4da3dee587 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:19:06.231558 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21795214-76de-426e-9741-ee4da3dee587 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:19:06.257845 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004168b7-0d70-4917-8961-2b6fe56e58f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:19:06.258731 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004168b7-0d70-4917-8961-2b6fe56e58f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:19:06.259191 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004168b7-0d70-4917-8961-2b6fe56e58f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:19:06.259353 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004168b7-0d70-4917-8961-2b6fe56e58f9 admin admin] PATH_INFO: `/v3/projects/c1a3a6fae6cf47a181dbec6349a501e5` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:19:06.262887 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004168b7-0d70-4917-8961-2b6fe56e58f9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c1a3a6fae6cf47a181dbec6349a501e5)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:19:06.265478 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004168b7-0d70-4917-8961-2b6fe56e58f9 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:19:06.442332 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-004168b7-0d70-4917-8961-2b6fe56e58f9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c1a3a6fae6cf47a181dbec6349a501e5', 'request_id': 'req-004168b7-0d70-4917-8961-2b6fe56e58f9'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:19:06.457450 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-004168b7-0d70-4917-8961-2b6fe56e58f9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c1a3a6fae6cf47a181dbec6349a501e5', 'request_id': 'req-004168b7-0d70-4917-8961-2b6fe56e58f9'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:19:06.485370 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-004168b7-0d70-4917-8961-2b6fe56e58f9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c1a3a6fae6cf47a181dbec6349a501e5', 'request_id': 'req-004168b7-0d70-4917-8961-2b6fe56e58f9'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:19:06.619633 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-004168b7-0d70-4917-8961-2b6fe56e58f9 admin admin] The token cache is being invalidate because project c1a3a6fae6cf47a181dbec6349a501e5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81677) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 10 12:19:06.619633 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-004168b7-0d70-4917-8961-2b6fe56e58f9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:19:06.619633 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-004168b7-0d70-4917-8961-2b6fe56e58f9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:19:06.621509 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1212/2402] 10.222.0.46 () {58 vars in 1233 bytes} [Mon Oct 10 12:19:06 2022] DELETE /identity/v3/projects/c1a3a6fae6cf47a181dbec6349a501e5 => generated 0 bytes in 420 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:19:06.626378 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe2dd96-5ef0-4dd6-b755-b6552e5185d2 service neutron] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:19:06.638117 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe2dd96-5ef0-4dd6-b755-b6552e5185d2 service neutron] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:19:06.728614 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a93aaf-a129-4f23-bc39-bf02c5982c21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:19:06.728614 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a93aaf-a129-4f23-bc39-bf02c5982c21 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:19:06.728614 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a93aaf-a129-4f23-bc39-bf02c5982c21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:19:06.728614 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a93aaf-a129-4f23-bc39-bf02c5982c21 admin admin] PATH_INFO: `/v3/users/ad60e58aad0c404198b2a7b0c4845b82` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:19:06.729175 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a93aaf-a129-4f23-bc39-bf02c5982c21 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ad60e58aad0c404198b2a7b0c4845b82)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:19:06.731350 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a93aaf-a129-4f23-bc39-bf02c5982c21 admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:19:06.926893 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:19:06.941191 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-41a93aaf-a129-4f23-bc39-bf02c5982c21 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ad60e58aad0c404198b2a7b0c4845b82'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:19:06.952948 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-41a93aaf-a129-4f23-bc39-bf02c5982c21 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ad60e58aad0c404198b2a7b0c4845b82'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:19:06.953939 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-41a93aaf-a129-4f23-bc39-bf02c5982c21 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ad60e58aad0c404198b2a7b0c4845b82'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:19:07.059066 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-41a93aaf-a129-4f23-bc39-bf02c5982c21 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ad60e58aad0c404198b2a7b0c4845b82'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:19:07.096683 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-41a93aaf-a129-4f23-bc39-bf02c5982c21 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ad60e58aad0c404198b2a7b0c4845b82'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:19:07.113550 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1191/2403] 10.222.0.46 () {58 vars in 1224 bytes} [Mon Oct 10 12:19:06 2022] DELETE /identity/v3/users/ad60e58aad0c404198b2a7b0c4845b82 => generated 0 bytes in 488 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:19:07.653160 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004168b7-0d70-4917-8961-2b6fe56e58f9 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:19:07.656093 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-004168b7-0d70-4917-8961-2b6fe56e58f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:19:07.696481 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004168b7-0d70-4917-8961-2b6fe56e58f9 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:19:07.862151 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f765841-ae0c-4ce2-a055-093b604efd77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:19:07.863595 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f765841-ae0c-4ce2-a055-093b604efd77 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:19:07.865425 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f765841-ae0c-4ce2-a055-093b604efd77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:19:07.867422 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f765841-ae0c-4ce2-a055-093b604efd77 admin admin] PATH_INFO: `/v3/projects/926987f9991e44d387b60d468e63c6ca` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:19:07.871579 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f765841-ae0c-4ce2-a055-093b604efd77 admin admin] RBAC: Authorizing `identity:delete_project(project_id=926987f9991e44d387b60d468e63c6ca)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:19:07.875166 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f765841-ae0c-4ce2-a055-093b604efd77 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:19:08.018153 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:19:08.065774 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-8f765841-ae0c-4ce2-a055-093b604efd77 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '926987f9991e44d387b60d468e63c6ca', 'request_id': 'req-8f765841-ae0c-4ce2-a055-093b604efd77'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:19:08.092328 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-8f765841-ae0c-4ce2-a055-093b604efd77 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '926987f9991e44d387b60d468e63c6ca', 'request_id': 'req-8f765841-ae0c-4ce2-a055-093b604efd77'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:19:08.106656 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-8f765841-ae0c-4ce2-a055-093b604efd77 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '926987f9991e44d387b60d468e63c6ca', 'request_id': 'req-8f765841-ae0c-4ce2-a055-093b604efd77'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:19:08.209306 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-8f765841-ae0c-4ce2-a055-093b604efd77 admin admin] The token cache is being invalidate because project 926987f9991e44d387b60d468e63c6ca was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81677) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 10 12:19:08.209721 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-8f765841-ae0c-4ce2-a055-093b604efd77 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:19:08.210575 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-8f765841-ae0c-4ce2-a055-093b604efd77 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:19:08.212696 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1213/2404] 10.222.0.46 () {58 vars in 1233 bytes} [Mon Oct 10 12:19:07 2022] DELETE /identity/v3/projects/926987f9991e44d387b60d468e63c6ca => generated 0 bytes in 560 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:19:08.218622 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f9efc8-0d19-40e9-bcd6-ca8eb9cbdcbe None None] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:19:08.219663 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f9efc8-0d19-40e9-bcd6-ca8eb9cbdcbe None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:19:08.219663 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f9efc8-0d19-40e9-bcd6-ca8eb9cbdcbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:19:08.251115 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.auth.core [None req-c8f9efc8-0d19-40e9-bcd6-ca8eb9cbdcbe None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:19:08.252703 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-c8f9efc8-0d19-40e9-bcd6-ca8eb9cbdcbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:19:08.356321 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1192/2405] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:19:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:19:08.368053 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80512e7-e6fb-42b4-9f9c-9307c0ef3655 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:19:08.368727 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80512e7-e6fb-42b4-9f9c-9307c0ef3655 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:19:08.369184 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80512e7-e6fb-42b4-9f9c-9307c0ef3655 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:19:08.392549 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-b80512e7-e6fb-42b4-9f9c-9307c0ef3655 None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:19:08.394122 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-b80512e7-e6fb-42b4-9f9c-9307c0ef3655 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:19:08.403941 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1214/2406] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:19:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:19:08.409747 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f9efc8-0d19-40e9-bcd6-ca8eb9cbdcbe None None] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:19:08.420699 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f9efc8-0d19-40e9-bcd6-ca8eb9cbdcbe None None] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:19:08.427181 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bed55b-bd4b-437b-9487-843111b6ce6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:19:08.427815 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bed55b-bd4b-437b-9487-843111b6ce6a admin admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:19:08.428094 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bed55b-bd4b-437b-9487-843111b6ce6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:19:08.428371 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bed55b-bd4b-437b-9487-843111b6ce6a admin admin] PATH_INFO: `/v3/domains` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:19:08.429078 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71bed55b-bd4b-437b-9487-843111b6ce6a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 12:19:08.429379 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71bed55b-bd4b-437b-9487-843111b6ce6a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:19:08.431037 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71bed55b-bd4b-437b-9487-843111b6ce6a admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:19:08.446789 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1193/2407] 10.222.0.46 () {58 vars in 1153 bytes} [Mon Oct 10 12:19:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 12:19:08.451545 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80512e7-e6fb-42b4-9f9c-9307c0ef3655 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:19:08.474661 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80512e7-e6fb-42b4-9f9c-9307c0ef3655 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:19:08.474661 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7b839f-2b3a-4ffa-8286-c156c7ff5a88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:19:08.474661 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7b839f-2b3a-4ffa-8286-c156c7ff5a88 admin admin] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:19:08.474661 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7b839f-2b3a-4ffa-8286-c156c7ff5a88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:19:08.476303 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7b839f-2b3a-4ffa-8286-c156c7ff5a88 admin admin] PATH_INFO: `/v3/projects` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:19:08.476303 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7b839f-2b3a-4ffa-8286-c156c7ff5a88 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:19:08.480683 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7b839f-2b3a-4ffa-8286-c156c7ff5a88 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:19:08.578169 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1215/2408] 10.222.0.46 () {60 vars in 1153 bytes} [Mon Oct 10 12:19:08 2022] POST /identity/v3/projects => generated 383 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 12:19:08.580012 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bed55b-bd4b-437b-9487-843111b6ce6a admin admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:19:08.594075 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bed55b-bd4b-437b-9487-843111b6ce6a admin admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:19:08.685838 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65882403-6451-46f9-93c3-c6ab49b1ee1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:19:08.686554 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65882403-6451-46f9-93c3-c6ab49b1ee1d admin admin] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:19:08.686873 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65882403-6451-46f9-93c3-c6ab49b1ee1d admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:19:08.687155 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65882403-6451-46f9-93c3-c6ab49b1ee1d admin admin] PATH_INFO: `/v3/users` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:19:08.687934 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65882403-6451-46f9-93c3-c6ab49b1ee1d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:19:08.690656 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65882403-6451-46f9-93c3-c6ab49b1ee1d admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:19:08.783609 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1194/2409] 10.222.0.46 () {60 vars in 1144 bytes} [Mon Oct 10 12:19:08 2022] POST /identity/v3/users => generated 292 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 12:19:08.784003 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7b839f-2b3a-4ffa-8286-c156c7ff5a88 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:19:08.799130 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7b839f-2b3a-4ffa-8286-c156c7ff5a88 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:19:08.806796 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ab360a-acf0-41f4-b327-4624c6027f5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:19:08.807828 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ab360a-acf0-41f4-b327-4624c6027f5f admin admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:19:08.808352 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ab360a-acf0-41f4-b327-4624c6027f5f admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:19:08.809365 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ab360a-acf0-41f4-b327-4624c6027f5f admin admin] PATH_INFO: `/v3/roles` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:19:08.810370 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ab360a-acf0-41f4-b327-4624c6027f5f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:19:08.812254 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ab360a-acf0-41f4-b327-4624c6027f5f admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:19:08.825478 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1216/2410] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 10 12:19:08 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 12:19:08.830419 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65882403-6451-46f9-93c3-c6ab49b1ee1d admin admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:19:08.844649 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65882403-6451-46f9-93c3-c6ab49b1ee1d admin admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:19:08.850579 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd04bebe-414c-407d-ae70-1c8e105a5140 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:19:08.851429 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd04bebe-414c-407d-ae70-1c8e105a5140 admin admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:19:08.851806 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd04bebe-414c-407d-ae70-1c8e105a5140 admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:19:08.852203 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd04bebe-414c-407d-ae70-1c8e105a5140 admin admin] PATH_INFO: `/v3/roles` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:19:08.852973 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd04bebe-414c-407d-ae70-1c8e105a5140 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:19:08.855037 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd04bebe-414c-407d-ae70-1c8e105a5140 admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:19:08.871709 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1195/2411] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 10 12:19:08 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 12:19:08.876669 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ab360a-acf0-41f4-b327-4624c6027f5f admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:19:08.893257 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ab360a-acf0-41f4-b327-4624c6027f5f admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:19:08.901211 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ec4699-5c68-4991-99a0-f5ca75856ffe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:19:08.902165 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ec4699-5c68-4991-99a0-f5ca75856ffe admin admin] REQUEST_METHOD: `PUT` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:19:08.902552 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ec4699-5c68-4991-99a0-f5ca75856ffe admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:19:08.902988 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ec4699-5c68-4991-99a0-f5ca75856ffe admin admin] PATH_INFO: `/v3/projects/b6a852ac65e94d3fae33ef19f5e6cefa/users/61e0f912b05140e9bb1a111193bb5056/roles/b2cb2a6e818e47338ce071e6fedfc673` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:19:08.926008 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ec4699-5c68-4991-99a0-f5ca75856ffe admin admin] RBAC: Authorizing `identity:create_grant(project_id=b6a852ac65e94d3fae33ef19f5e6cefa, user_id=61e0f912b05140e9bb1a111193bb5056, role_id=b2cb2a6e818e47338ce071e6fedfc673)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:19:08.929323 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ec4699-5c68-4991-99a0-f5ca75856ffe admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:19:09.003272 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1217/2412] 10.222.0.46 () {60 vars in 1483 bytes} [Mon Oct 10 12:19:08 2022] PUT /identity/v3/projects/b6a852ac65e94d3fae33ef19f5e6cefa/users/61e0f912b05140e9bb1a111193bb5056/roles/b2cb2a6e818e47338ce071e6fedfc673 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:19:09.103516 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc9dc92-e4a7-4405-a08f-bda1c7a6e6b4 None None] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:19:09.103901 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc9dc92-e4a7-4405-a08f-bda1c7a6e6b4 None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:19:09.104154 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc9dc92-e4a7-4405-a08f-bda1c7a6e6b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:19:09.174821 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.auth.core [None req-9fc9dc92-e4a7-4405-a08f-bda1c7a6e6b4 None None] MFA Rules not processed for user `61e0f912b05140e9bb1a111193bb5056`. Rule list: `[]` (Enabled: `True`). {{(pid=81676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:19:09.176350 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-9fc9dc92-e4a7-4405-a08f-bda1c7a6e6b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:19:09.183723 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:19:09.308572 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1196/2413] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:19:09 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:19:09.313755 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ec4699-5c68-4991-99a0-f5ca75856ffe admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:19:09.384766 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ec4699-5c68-4991-99a0-f5ca75856ffe admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:19:09.389336 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51340006-af61-4463-b0f8-9b9b2248707e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:19:09.389943 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51340006-af61-4463-b0f8-9b9b2248707e admin admin] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:19:09.390227 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51340006-af61-4463-b0f8-9b9b2248707e admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:19:09.390501 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51340006-af61-4463-b0f8-9b9b2248707e admin admin] PATH_INFO: `/v3/projects` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:19:09.391183 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51340006-af61-4463-b0f8-9b9b2248707e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:19:09.393083 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51340006-af61-4463-b0f8-9b9b2248707e admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:19:09.687744 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1218/2414] 10.222.0.46 () {60 vars in 1153 bytes} [Mon Oct 10 12:19:09 2022] POST /identity/v3/projects => generated 381 bytes in 375 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 12:19:09.692364 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc9dc92-e4a7-4405-a08f-bda1c7a6e6b4 None None] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:19:09.707314 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc9dc92-e4a7-4405-a08f-bda1c7a6e6b4 None None] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:19:09.775628 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98c4698-e9e0-4f98-9df8-7c90a6808ad8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:19:09.776437 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98c4698-e9e0-4f98-9df8-7c90a6808ad8 admin admin] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:19:09.776817 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98c4698-e9e0-4f98-9df8-7c90a6808ad8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:19:09.777186 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98c4698-e9e0-4f98-9df8-7c90a6808ad8 admin admin] PATH_INFO: `/v3/users` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:19:09.778029 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98c4698-e9e0-4f98-9df8-7c90a6808ad8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:19:09.780699 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98c4698-e9e0-4f98-9df8-7c90a6808ad8 admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:19:09.937947 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1197/2415] 10.222.0.46 () {60 vars in 1144 bytes} [Mon Oct 10 12:19:09 2022] POST /identity/v3/users => generated 297 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 12:19:09.941963 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51340006-af61-4463-b0f8-9b9b2248707e admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:19:09.953604 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51340006-af61-4463-b0f8-9b9b2248707e admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:19:09.958524 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6ebb41-0927-485c-8be1-ee6b566d1b4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:19:09.959197 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6ebb41-0927-485c-8be1-ee6b566d1b4e admin admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:19:09.959480 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6ebb41-0927-485c-8be1-ee6b566d1b4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:19:09.960195 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6ebb41-0927-485c-8be1-ee6b566d1b4e admin admin] PATH_INFO: `/v3/roles` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:19:09.960489 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6ebb41-0927-485c-8be1-ee6b566d1b4e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:19:09.962141 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6ebb41-0927-485c-8be1-ee6b566d1b4e admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:19:09.974908 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1219/2416] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 10 12:19:09 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 12:19:09.979142 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98c4698-e9e0-4f98-9df8-7c90a6808ad8 admin admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:19:09.989799 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98c4698-e9e0-4f98-9df8-7c90a6808ad8 admin admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:19:09.994486 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1cc4f7f-8b26-476c-ab19-2071b99c1edb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:19:09.995169 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cc4f7f-8b26-476c-ab19-2071b99c1edb admin admin] REQUEST_METHOD: `PUT` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:19:09.995451 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cc4f7f-8b26-476c-ab19-2071b99c1edb admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:19:09.995726 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cc4f7f-8b26-476c-ab19-2071b99c1edb admin admin] PATH_INFO: `/v3/projects/794da89ca3414671a57ad662eccb21b7/users/33c2f6c577a746f8b765af01c820dcce/roles/0d822a0674b6495fac623877a3b38025` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:19:10.018638 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1cc4f7f-8b26-476c-ab19-2071b99c1edb admin admin] RBAC: Authorizing `identity:create_grant(project_id=794da89ca3414671a57ad662eccb21b7, user_id=33c2f6c577a746f8b765af01c820dcce, role_id=0d822a0674b6495fac623877a3b38025)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:19:10.018638 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1cc4f7f-8b26-476c-ab19-2071b99c1edb admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:19:10.133119 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1198/2417] 10.222.0.46 () {60 vars in 1483 bytes} [Mon Oct 10 12:19:09 2022] PUT /identity/v3/projects/794da89ca3414671a57ad662eccb21b7/users/33c2f6c577a746f8b765af01c820dcce/roles/0d822a0674b6495fac623877a3b38025 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:19:10.137744 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6ebb41-0927-485c-8be1-ee6b566d1b4e admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:19:10.161973 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6ebb41-0927-485c-8be1-ee6b566d1b4e admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:19:10.198001 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:19:10.270139 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe6baac-2722-49d2-b289-ccae54a72df1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:19:10.271412 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe6baac-2722-49d2-b289-ccae54a72df1 admin admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:19:10.277192 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe6baac-2722-49d2-b289-ccae54a72df1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:19:10.277192 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe6baac-2722-49d2-b289-ccae54a72df1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:19:10.277192 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe6baac-2722-49d2-b289-ccae54a72df1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:19:10.277192 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe6baac-2722-49d2-b289-ccae54a72df1 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:19:10.291424 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1220/2418] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 10 12:19:10 2022] GET /identity/v3/roles => generated 1420 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 12:19:10.299968 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1cc4f7f-8b26-476c-ab19-2071b99c1edb admin admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:19:10.311987 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1cc4f7f-8b26-476c-ab19-2071b99c1edb admin admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:19:10.317319 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb819dd-9729-45af-b1c2-96ab2b272396 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:19:10.318266 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb819dd-9729-45af-b1c2-96ab2b272396 admin admin] REQUEST_METHOD: `PUT` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:19:10.318547 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb819dd-9729-45af-b1c2-96ab2b272396 admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:19:10.318952 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb819dd-9729-45af-b1c2-96ab2b272396 admin admin] PATH_INFO: `/v3/projects/794da89ca3414671a57ad662eccb21b7/users/33c2f6c577a746f8b765af01c820dcce/roles/0d822a0674b6495fac623877a3b38025` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:19:10.321690 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb819dd-9729-45af-b1c2-96ab2b272396 admin admin] RBAC: Authorizing `identity:create_grant(project_id=794da89ca3414671a57ad662eccb21b7, user_id=33c2f6c577a746f8b765af01c820dcce, role_id=0d822a0674b6495fac623877a3b38025)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:19:10.324711 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb819dd-9729-45af-b1c2-96ab2b272396 admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:19:10.337623 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1199/2419] 10.222.0.46 () {60 vars in 1483 bytes} [Mon Oct 10 12:19:10 2022] PUT /identity/v3/projects/794da89ca3414671a57ad662eccb21b7/users/33c2f6c577a746f8b765af01c820dcce/roles/0d822a0674b6495fac623877a3b38025 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:19:10.348343 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cf358f-64df-42c5-b844-127aff2c1913 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:19:10.348849 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cf358f-64df-42c5-b844-127aff2c1913 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:19:10.349241 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cf358f-64df-42c5-b844-127aff2c1913 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:19:10.399226 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-a5cf358f-64df-42c5-b844-127aff2c1913 None None] MFA Rules not processed for user `33c2f6c577a746f8b765af01c820dcce`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:19:10.400749 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-a5cf358f-64df-42c5-b844-127aff2c1913 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:19:10.538139 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1221/2420] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:19:10 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:19:10.550053 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb819dd-9729-45af-b1c2-96ab2b272396 admin admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:19:10.551906 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-3eb819dd-9729-45af-b1c2-96ab2b272396 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:19:10.633103 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb819dd-9729-45af-b1c2-96ab2b272396 admin admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:19:10.637879 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98b26ef-f44d-4b3d-9bb2-8d69c43fd1c3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3ae093470484b93b23dcfae1657df9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad3772f7b6a47c28b7ba37a9316a3c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:19:10.639366 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98b26ef-f44d-4b3d-9bb2-8d69c43fd1c3 service cinder] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:19:10.639754 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98b26ef-f44d-4b3d-9bb2-8d69c43fd1c3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:19:10.640109 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98b26ef-f44d-4b3d-9bb2-8d69c43fd1c3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:19:10.657803 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98b26ef-f44d-4b3d-9bb2-8d69c43fd1c3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:19:10.661428 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98b26ef-f44d-4b3d-9bb2-8d69c43fd1c3 service cinder] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:19:10.672742 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1200/2421] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 10 12:19:10 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 10 12:19:10.897577 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5cf358f-64df-42c5-b844-127aff2c1913 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:19:10.908162 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5cf358f-64df-42c5-b844-127aff2c1913 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:19:10.912681 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec0b512-ff7e-4e79-9348-dc2a8abc2215 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3ae093470484b93b23dcfae1657df9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad3772f7b6a47c28b7ba37a9316a3c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:19:10.913362 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec0b512-ff7e-4e79-9348-dc2a8abc2215 service cinder] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:19:10.913741 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec0b512-ff7e-4e79-9348-dc2a8abc2215 service cinder] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:19:10.914069 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec0b512-ff7e-4e79-9348-dc2a8abc2215 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:19:10.982826 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec0b512-ff7e-4e79-9348-dc2a8abc2215 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:19:10.984268 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec0b512-ff7e-4e79-9348-dc2a8abc2215 service cinder] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:19:10.995068 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1222/2422] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 10 12:19:10 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:19:11.520886 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98b26ef-f44d-4b3d-9bb2-8d69c43fd1c3 service cinder] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:19:11.526912 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:19:11.623542 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98b26ef-f44d-4b3d-9bb2-8d69c43fd1c3 service cinder] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:19:11.628875 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39cf2938-62c0-4fab-b3d2-08999f850077 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:19:11.629748 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cf2938-62c0-4fab-b3d2-08999f850077 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:19:11.630181 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cf2938-62c0-4fab-b3d2-08999f850077 admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:19:11.630474 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cf2938-62c0-4fab-b3d2-08999f850077 admin admin] PATH_INFO: `/v3/users/61e0f912b05140e9bb1a111193bb5056` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:19:11.632863 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39cf2938-62c0-4fab-b3d2-08999f850077 admin admin] RBAC: Authorizing `identity:delete_user(user_id=61e0f912b05140e9bb1a111193bb5056)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:19:11.637109 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39cf2938-62c0-4fab-b3d2-08999f850077 admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:19:11.893301 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-39cf2938-62c0-4fab-b3d2-08999f850077 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '61e0f912b05140e9bb1a111193bb5056'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:19:11.902870 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-39cf2938-62c0-4fab-b3d2-08999f850077 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '61e0f912b05140e9bb1a111193bb5056'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:19:11.904336 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-39cf2938-62c0-4fab-b3d2-08999f850077 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '61e0f912b05140e9bb1a111193bb5056'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:19:12.000317 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-39cf2938-62c0-4fab-b3d2-08999f850077 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '61e0f912b05140e9bb1a111193bb5056'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:19:12.025123 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-39cf2938-62c0-4fab-b3d2-08999f850077 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '61e0f912b05140e9bb1a111193bb5056'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:19:12.036572 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1201/2423] 10.222.0.46 () {58 vars in 1224 bytes} [Mon Oct 10 12:19:11 2022] DELETE /identity/v3/users/61e0f912b05140e9bb1a111193bb5056 => generated 0 bytes in 514 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:19:12.038233 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33851bd-2b05-4d2f-a481-4b7ab2ca86ab None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:19:12.038988 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33851bd-2b05-4d2f-a481-4b7ab2ca86ab None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:19:12.038988 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33851bd-2b05-4d2f-a481-4b7ab2ca86ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:19:12.067727 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-b33851bd-2b05-4d2f-a481-4b7ab2ca86ab None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:19:12.069185 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-b33851bd-2b05-4d2f-a481-4b7ab2ca86ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:19:12.118935 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:19:12.145213 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1223/2424] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:19:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:19:12.155633 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39cf2938-62c0-4fab-b3d2-08999f850077 admin admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:19:12.156975 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-39cf2938-62c0-4fab-b3d2-08999f850077 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:19:12.229895 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39cf2938-62c0-4fab-b3d2-08999f850077 admin admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:19:12.234853 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4563a91-c7a5-410c-a5b9-f6712fba7f25 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58b1c7c85302421cba823ace4201861f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad3772f7b6a47c28b7ba37a9316a3c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:19:12.235463 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4563a91-c7a5-410c-a5b9-f6712fba7f25 service neutron] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:19:12.235753 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4563a91-c7a5-410c-a5b9-f6712fba7f25 service neutron] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:19:12.236062 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4563a91-c7a5-410c-a5b9-f6712fba7f25 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:19:12.246854 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4563a91-c7a5-410c-a5b9-f6712fba7f25 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:19:12.249057 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4563a91-c7a5-410c-a5b9-f6712fba7f25 service neutron] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:19:12.259380 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1202/2425] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 10 12:19:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:19:12.787234 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33851bd-2b05-4d2f-a481-4b7ab2ca86ab None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:19:12.788680 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-b33851bd-2b05-4d2f-a481-4b7ab2ca86ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:19:12.810998 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33851bd-2b05-4d2f-a481-4b7ab2ca86ab None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:19:12.816950 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f21bb4-fab9-4dbe-83ee-56922b793aa3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:19:12.817620 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f21bb4-fab9-4dbe-83ee-56922b793aa3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:19:12.817900 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f21bb4-fab9-4dbe-83ee-56922b793aa3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:19:12.818182 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f21bb4-fab9-4dbe-83ee-56922b793aa3 admin admin] PATH_INFO: `/v3/projects/b6a852ac65e94d3fae33ef19f5e6cefa` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:19:12.819370 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f21bb4-fab9-4dbe-83ee-56922b793aa3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b6a852ac65e94d3fae33ef19f5e6cefa)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:19:12.821306 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f21bb4-fab9-4dbe-83ee-56922b793aa3 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:19:13.015604 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-62f21bb4-fab9-4dbe-83ee-56922b793aa3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b6a852ac65e94d3fae33ef19f5e6cefa', 'request_id': 'req-62f21bb4-fab9-4dbe-83ee-56922b793aa3'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:19:13.022589 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-62f21bb4-fab9-4dbe-83ee-56922b793aa3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b6a852ac65e94d3fae33ef19f5e6cefa', 'request_id': 'req-62f21bb4-fab9-4dbe-83ee-56922b793aa3'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:19:13.031770 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-62f21bb4-fab9-4dbe-83ee-56922b793aa3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b6a852ac65e94d3fae33ef19f5e6cefa', 'request_id': 'req-62f21bb4-fab9-4dbe-83ee-56922b793aa3'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:19:13.118353 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-62f21bb4-fab9-4dbe-83ee-56922b793aa3 admin admin] The token cache is being invalidate because project b6a852ac65e94d3fae33ef19f5e6cefa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81677) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 10 12:19:13.119028 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-62f21bb4-fab9-4dbe-83ee-56922b793aa3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:19:13.119887 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-62f21bb4-fab9-4dbe-83ee-56922b793aa3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:19:13.126141 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4563a91-c7a5-410c-a5b9-f6712fba7f25 service neutron] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:19:13.132597 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1224/2426] 10.222.0.46 () {58 vars in 1233 bytes} [Mon Oct 10 12:19:12 2022] DELETE /identity/v3/projects/b6a852ac65e94d3fae33ef19f5e6cefa => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:19:13.146719 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4563a91-c7a5-410c-a5b9-f6712fba7f25 service neutron] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:19:13.192238 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:19:13.268094 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd19cf3-1d93-4473-809d-d7f05a7e5e14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:19:13.268821 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd19cf3-1d93-4473-809d-d7f05a7e5e14 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:19:13.269226 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd19cf3-1d93-4473-809d-d7f05a7e5e14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:19:13.290995 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd19cf3-1d93-4473-809d-d7f05a7e5e14 admin admin] PATH_INFO: `/v3/users/33c2f6c577a746f8b765af01c820dcce` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:19:13.290995 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd19cf3-1d93-4473-809d-d7f05a7e5e14 admin admin] RBAC: Authorizing `identity:delete_user(user_id=33c2f6c577a746f8b765af01c820dcce)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:19:13.290995 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd19cf3-1d93-4473-809d-d7f05a7e5e14 admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:19:13.824606 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-4dd19cf3-1d93-4473-809d-d7f05a7e5e14 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '33c2f6c577a746f8b765af01c820dcce'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:19:13.834792 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-4dd19cf3-1d93-4473-809d-d7f05a7e5e14 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '33c2f6c577a746f8b765af01c820dcce'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:19:13.835564 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-4dd19cf3-1d93-4473-809d-d7f05a7e5e14 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '33c2f6c577a746f8b765af01c820dcce'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:19:13.977147 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-4dd19cf3-1d93-4473-809d-d7f05a7e5e14 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '33c2f6c577a746f8b765af01c820dcce'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:19:13.992032 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-4dd19cf3-1d93-4473-809d-d7f05a7e5e14 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '33c2f6c577a746f8b765af01c820dcce'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:19:13.999104 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1203/2427] 10.222.0.46 () {58 vars in 1224 bytes} [Mon Oct 10 12:19:13 2022] DELETE /identity/v3/users/33c2f6c577a746f8b765af01c820dcce => generated 0 bytes in 874 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:19:15.033150 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f21bb4-fab9-4dbe-83ee-56922b793aa3 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:19:15.034477 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-62f21bb4-fab9-4dbe-83ee-56922b793aa3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:19:15.049656 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f21bb4-fab9-4dbe-83ee-56922b793aa3 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:19:15.151276 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089098c5-0688-4f8b-8e15-d9726d7597cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:19:15.152429 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089098c5-0688-4f8b-8e15-d9726d7597cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:19:15.152801 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089098c5-0688-4f8b-8e15-d9726d7597cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:19:15.153180 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089098c5-0688-4f8b-8e15-d9726d7597cd admin admin] PATH_INFO: `/v3/projects/794da89ca3414671a57ad662eccb21b7` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:19:15.155336 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089098c5-0688-4f8b-8e15-d9726d7597cd admin admin] RBAC: Authorizing `identity:delete_project(project_id=794da89ca3414671a57ad662eccb21b7)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:19:15.157386 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089098c5-0688-4f8b-8e15-d9726d7597cd admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:19:15.340820 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-089098c5-0688-4f8b-8e15-d9726d7597cd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '794da89ca3414671a57ad662eccb21b7', 'request_id': 'req-089098c5-0688-4f8b-8e15-d9726d7597cd'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:19:15.356381 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-089098c5-0688-4f8b-8e15-d9726d7597cd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '794da89ca3414671a57ad662eccb21b7', 'request_id': 'req-089098c5-0688-4f8b-8e15-d9726d7597cd'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:19:15.370607 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-089098c5-0688-4f8b-8e15-d9726d7597cd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '794da89ca3414671a57ad662eccb21b7', 'request_id': 'req-089098c5-0688-4f8b-8e15-d9726d7597cd'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:19:15.518123 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:19:15.523953 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-089098c5-0688-4f8b-8e15-d9726d7597cd admin admin] The token cache is being invalidate because project 794da89ca3414671a57ad662eccb21b7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81677) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 10 12:19:15.525596 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-089098c5-0688-4f8b-8e15-d9726d7597cd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:19:15.525596 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-089098c5-0688-4f8b-8e15-d9726d7597cd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:19:15.528213 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1225/2428] 10.222.0.46 () {58 vars in 1233 bytes} [Mon Oct 10 12:19:15 2022] DELETE /identity/v3/projects/794da89ca3414671a57ad662eccb21b7 => generated 0 bytes in 495 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:19:15.533233 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4dccf57-035a-424f-ae2e-248d52e4e05b None None] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:19:15.533986 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4dccf57-035a-424f-ae2e-248d52e4e05b None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:19:15.534339 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4dccf57-035a-424f-ae2e-248d52e4e05b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:19:15.569996 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.auth.core [None req-f4dccf57-035a-424f-ae2e-248d52e4e05b None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:19:15.569996 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-f4dccf57-035a-424f-ae2e-248d52e4e05b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:19:15.645906 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1204/2429] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:19:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:19:15.658152 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffc49e9-f659-4a45-938e-98292718a6e5 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:19:15.658631 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffc49e9-f659-4a45-938e-98292718a6e5 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:19:15.659161 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffc49e9-f659-4a45-938e-98292718a6e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:19:15.695890 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-8ffc49e9-f659-4a45-938e-98292718a6e5 None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:19:15.697151 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-8ffc49e9-f659-4a45-938e-98292718a6e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:19:15.706011 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1226/2430] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:19:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:19:15.710309 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4dccf57-035a-424f-ae2e-248d52e4e05b None None] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:19:15.722274 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4dccf57-035a-424f-ae2e-248d52e4e05b None None] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:19:15.728957 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a3140d-2bd6-4035-bf84-f16e27c352ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:19:15.733681 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a3140d-2bd6-4035-bf84-f16e27c352ee admin admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:19:15.733681 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a3140d-2bd6-4035-bf84-f16e27c352ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:19:15.733681 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a3140d-2bd6-4035-bf84-f16e27c352ee admin admin] PATH_INFO: `/v3/domains` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:19:15.733681 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a3140d-2bd6-4035-bf84-f16e27c352ee admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 12:19:15.733681 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a3140d-2bd6-4035-bf84-f16e27c352ee admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:19:15.733681 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a3140d-2bd6-4035-bf84-f16e27c352ee admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:19:15.746563 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1205/2431] 10.222.0.46 () {58 vars in 1153 bytes} [Mon Oct 10 12:19:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 12:19:15.752307 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffc49e9-f659-4a45-938e-98292718a6e5 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:19:15.761620 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffc49e9-f659-4a45-938e-98292718a6e5 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:19:15.767090 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074308b6-d1d0-43f1-b8a0-e1ab00e5f939 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:19:15.767714 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074308b6-d1d0-43f1-b8a0-e1ab00e5f939 admin admin] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:19:15.768118 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074308b6-d1d0-43f1-b8a0-e1ab00e5f939 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:19:15.768357 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074308b6-d1d0-43f1-b8a0-e1ab00e5f939 admin admin] PATH_INFO: `/v3/projects` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:19:15.768908 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074308b6-d1d0-43f1-b8a0-e1ab00e5f939 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:19:15.770863 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074308b6-d1d0-43f1-b8a0-e1ab00e5f939 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:19:15.907146 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1227/2432] 10.222.0.46 () {60 vars in 1153 bytes} [Mon Oct 10 12:19:15 2022] POST /identity/v3/projects => generated 371 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 12:19:15.908483 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a3140d-2bd6-4035-bf84-f16e27c352ee admin admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:19:15.923283 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a3140d-2bd6-4035-bf84-f16e27c352ee admin admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:19:15.969051 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:19:16.002045 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1c29af-ca98-4509-9747-7e5161decebd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:19:16.002654 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1c29af-ca98-4509-9747-7e5161decebd admin admin] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:19:16.002978 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1c29af-ca98-4509-9747-7e5161decebd admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:19:16.003252 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1c29af-ca98-4509-9747-7e5161decebd admin admin] PATH_INFO: `/v3/users` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:19:16.004357 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1c29af-ca98-4509-9747-7e5161decebd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:19:16.005841 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1c29af-ca98-4509-9747-7e5161decebd admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:19:16.128724 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1206/2433] 10.222.0.46 () {60 vars in 1144 bytes} [Mon Oct 10 12:19:15 2022] POST /identity/v3/users => generated 286 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 12:19:16.132108 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074308b6-d1d0-43f1-b8a0-e1ab00e5f939 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:19:16.144277 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074308b6-d1d0-43f1-b8a0-e1ab00e5f939 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:19:16.149914 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af925725-b701-4ede-966f-079e53a2da46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:19:16.150683 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af925725-b701-4ede-966f-079e53a2da46 admin admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:19:16.151033 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af925725-b701-4ede-966f-079e53a2da46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:19:16.151368 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af925725-b701-4ede-966f-079e53a2da46 admin admin] PATH_INFO: `/v3/roles` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:19:16.152116 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af925725-b701-4ede-966f-079e53a2da46 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:19:16.153831 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af925725-b701-4ede-966f-079e53a2da46 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:19:16.165495 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1228/2434] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 10 12:19:16 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 12:19:16.170351 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1c29af-ca98-4509-9747-7e5161decebd admin admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:19:16.181041 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1c29af-ca98-4509-9747-7e5161decebd admin admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:19:16.186422 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2832ed6e-f3f8-49fa-be3f-4c7917748ede admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:19:16.187223 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2832ed6e-f3f8-49fa-be3f-4c7917748ede admin admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:19:16.187494 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2832ed6e-f3f8-49fa-be3f-4c7917748ede admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:19:16.187768 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2832ed6e-f3f8-49fa-be3f-4c7917748ede admin admin] PATH_INFO: `/v3/roles` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:19:16.188409 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2832ed6e-f3f8-49fa-be3f-4c7917748ede admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:19:16.190016 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2832ed6e-f3f8-49fa-be3f-4c7917748ede admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:19:16.200476 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1207/2435] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 10 12:19:16 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 12:19:16.204990 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af925725-b701-4ede-966f-079e53a2da46 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:19:16.215948 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af925725-b701-4ede-966f-079e53a2da46 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:19:16.222161 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7508d4-6b3c-4dec-bb53-6c25ba22dff8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:19:16.223339 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7508d4-6b3c-4dec-bb53-6c25ba22dff8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:19:16.223851 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7508d4-6b3c-4dec-bb53-6c25ba22dff8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:19:16.224280 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7508d4-6b3c-4dec-bb53-6c25ba22dff8 admin admin] PATH_INFO: `/v3/projects/c3bfb7f39ae94974bf42d53480a5273a/users/24b3c0eccddf41679fea5f7c953f4fcd/roles/b2cb2a6e818e47338ce071e6fedfc673` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:19:16.241774 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7508d4-6b3c-4dec-bb53-6c25ba22dff8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c3bfb7f39ae94974bf42d53480a5273a, user_id=24b3c0eccddf41679fea5f7c953f4fcd, role_id=b2cb2a6e818e47338ce071e6fedfc673)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:19:16.244456 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7508d4-6b3c-4dec-bb53-6c25ba22dff8 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:19:16.410239 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1229/2436] 10.222.0.46 () {60 vars in 1483 bytes} [Mon Oct 10 12:19:16 2022] PUT /identity/v3/projects/c3bfb7f39ae94974bf42d53480a5273a/users/24b3c0eccddf41679fea5f7c953f4fcd/roles/b2cb2a6e818e47338ce071e6fedfc673 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:19:16.421294 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210ded83-a500-49cb-b992-90f3f5b2cff2 None None] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:19:16.422010 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210ded83-a500-49cb-b992-90f3f5b2cff2 None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:19:16.422010 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210ded83-a500-49cb-b992-90f3f5b2cff2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:19:16.453892 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.auth.core [None req-210ded83-a500-49cb-b992-90f3f5b2cff2 None None] MFA Rules not processed for user `24b3c0eccddf41679fea5f7c953f4fcd`. Rule list: `[]` (Enabled: `True`). {{(pid=81676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:19:16.455182 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-210ded83-a500-49cb-b992-90f3f5b2cff2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:19:16.553287 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1208/2437] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:19:16 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:19:16.570137 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7508d4-6b3c-4dec-bb53-6c25ba22dff8 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:19:16.572052 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-ec7508d4-6b3c-4dec-bb53-6c25ba22dff8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:19:16.701755 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7508d4-6b3c-4dec-bb53-6c25ba22dff8 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:19:16.705634 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65959343-bc9c-47f0-b174-a5ebe9d6023e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3ae093470484b93b23dcfae1657df9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad3772f7b6a47c28b7ba37a9316a3c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:19:16.706256 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65959343-bc9c-47f0-b174-a5ebe9d6023e service cinder] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:19:16.706536 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65959343-bc9c-47f0-b174-a5ebe9d6023e service cinder] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:19:16.706969 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65959343-bc9c-47f0-b174-a5ebe9d6023e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:19:16.716995 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65959343-bc9c-47f0-b174-a5ebe9d6023e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:19:16.718438 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65959343-bc9c-47f0-b174-a5ebe9d6023e service cinder] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:19:16.728715 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1230/2438] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 10 12:19:16 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:19:16.756105 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210ded83-a500-49cb-b992-90f3f5b2cff2 None None] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:19:16.823114 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210ded83-a500-49cb-b992-90f3f5b2cff2 None None] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:19:16.828176 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6dce60-4540-41b5-be9e-14a184b3140d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:19:16.828882 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6dce60-4540-41b5-be9e-14a184b3140d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:19:16.829157 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6dce60-4540-41b5-be9e-14a184b3140d admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:19:16.829428 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6dce60-4540-41b5-be9e-14a184b3140d admin admin] PATH_INFO: `/v3/users/24b3c0eccddf41679fea5f7c953f4fcd` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:19:16.830555 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6dce60-4540-41b5-be9e-14a184b3140d admin admin] RBAC: Authorizing `identity:delete_user(user_id=24b3c0eccddf41679fea5f7c953f4fcd)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:19:16.832555 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6dce60-4540-41b5-be9e-14a184b3140d admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:19:17.014787 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=91,keystone:INSERT=3,keystone:DELETE=6 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:19:17.017711 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-8a6dce60-4540-41b5-be9e-14a184b3140d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '24b3c0eccddf41679fea5f7c953f4fcd'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:19:17.025835 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-8a6dce60-4540-41b5-be9e-14a184b3140d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '24b3c0eccddf41679fea5f7c953f4fcd'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:19:17.026711 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-8a6dce60-4540-41b5-be9e-14a184b3140d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '24b3c0eccddf41679fea5f7c953f4fcd'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:19:17.115855 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-8a6dce60-4540-41b5-be9e-14a184b3140d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '24b3c0eccddf41679fea5f7c953f4fcd'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:19:17.130478 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-8a6dce60-4540-41b5-be9e-14a184b3140d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '24b3c0eccddf41679fea5f7c953f4fcd'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:19:17.137029 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1209/2439] 10.222.0.46 () {58 vars in 1224 bytes} [Mon Oct 10 12:19:16 2022] DELETE /identity/v3/users/24b3c0eccddf41679fea5f7c953f4fcd => generated 0 bytes in 382 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:19:17.141579 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303537d0-1856-4f3f-8203-8d795ebcbfed None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:19:17.142091 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303537d0-1856-4f3f-8203-8d795ebcbfed None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:19:17.142230 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303537d0-1856-4f3f-8203-8d795ebcbfed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:19:17.162341 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-303537d0-1856-4f3f-8203-8d795ebcbfed None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:19:17.163684 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-303537d0-1856-4f3f-8203-8d795ebcbfed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:19:17.229738 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1231/2440] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:19:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 10 12:19:17.240464 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6dce60-4540-41b5-be9e-14a184b3140d admin admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:19:17.241887 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-8a6dce60-4540-41b5-be9e-14a184b3140d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:19:17.300549 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6dce60-4540-41b5-be9e-14a184b3140d admin admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:19:17.304290 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a044085-2c35-4e10-bbfb-9067a511f907 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58b1c7c85302421cba823ace4201861f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad3772f7b6a47c28b7ba37a9316a3c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:19:17.304898 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a044085-2c35-4e10-bbfb-9067a511f907 service neutron] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:19:17.305185 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a044085-2c35-4e10-bbfb-9067a511f907 service neutron] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:19:17.305538 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a044085-2c35-4e10-bbfb-9067a511f907 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:19:17.314837 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a044085-2c35-4e10-bbfb-9067a511f907 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:19:17.316322 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a044085-2c35-4e10-bbfb-9067a511f907 service neutron] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:19:17.326618 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1210/2441] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 10 12:19:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:19:17.767945 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303537d0-1856-4f3f-8203-8d795ebcbfed None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:19:17.769570 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-303537d0-1856-4f3f-8203-8d795ebcbfed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:19:17.782548 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303537d0-1856-4f3f-8203-8d795ebcbfed None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:19:17.789037 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de7c717-8111-4448-9fbc-2309f84bace9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:19:17.790879 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de7c717-8111-4448-9fbc-2309f84bace9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:19:17.791286 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de7c717-8111-4448-9fbc-2309f84bace9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:19:17.791812 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de7c717-8111-4448-9fbc-2309f84bace9 admin admin] PATH_INFO: `/v3/projects/c3bfb7f39ae94974bf42d53480a5273a` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:19:17.793554 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de7c717-8111-4448-9fbc-2309f84bace9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c3bfb7f39ae94974bf42d53480a5273a)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:19:17.796326 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de7c717-8111-4448-9fbc-2309f84bace9 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:19:17.887691 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=96,keystone:INSERT=2,keystone:DELETE=2 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:19:17.900338 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-8de7c717-8111-4448-9fbc-2309f84bace9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c3bfb7f39ae94974bf42d53480a5273a', 'request_id': 'req-8de7c717-8111-4448-9fbc-2309f84bace9'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:19:17.905929 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-8de7c717-8111-4448-9fbc-2309f84bace9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c3bfb7f39ae94974bf42d53480a5273a', 'request_id': 'req-8de7c717-8111-4448-9fbc-2309f84bace9'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:19:17.912113 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-8de7c717-8111-4448-9fbc-2309f84bace9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c3bfb7f39ae94974bf42d53480a5273a', 'request_id': 'req-8de7c717-8111-4448-9fbc-2309f84bace9'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:19:18.046979 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-8de7c717-8111-4448-9fbc-2309f84bace9 admin admin] The token cache is being invalidate because project c3bfb7f39ae94974bf42d53480a5273a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81677) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 10 12:19:18.047194 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-8de7c717-8111-4448-9fbc-2309f84bace9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:19:18.047820 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-8de7c717-8111-4448-9fbc-2309f84bace9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:19:18.049301 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1232/2442] 10.222.0.46 () {58 vars in 1233 bytes} [Mon Oct 10 12:19:17 2022] DELETE /identity/v3/projects/c3bfb7f39ae94974bf42d53480a5273a => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:19:18.054505 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d97bc0b-67c2-444f-b7d0-90b5091694dc None None] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:19:18.055039 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d97bc0b-67c2-444f-b7d0-90b5091694dc None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:19:18.055140 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d97bc0b-67c2-444f-b7d0-90b5091694dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:19:18.075253 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.auth.core [None req-9d97bc0b-67c2-444f-b7d0-90b5091694dc None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:19:18.077040 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-9d97bc0b-67c2-444f-b7d0-90b5091694dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:19:18.138015 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1211/2443] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:19:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:19:18.145789 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2028072-18ae-4631-a654-ad182b772e00 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:19:18.146289 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2028072-18ae-4631-a654-ad182b772e00 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:19:18.146390 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2028072-18ae-4631-a654-ad182b772e00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:19:18.166396 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-f2028072-18ae-4631-a654-ad182b772e00 None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:19:18.167653 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-f2028072-18ae-4631-a654-ad182b772e00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:19:18.176510 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1233/2444] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:19:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:19:18.181169 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d97bc0b-67c2-444f-b7d0-90b5091694dc None None] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:19:18.192890 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d97bc0b-67c2-444f-b7d0-90b5091694dc None None] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:19:18.197820 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b674b8-7bb3-4dd6-92aa-7e7f1c30dc98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:19:18.198320 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b674b8-7bb3-4dd6-92aa-7e7f1c30dc98 admin admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:19:18.198687 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b674b8-7bb3-4dd6-92aa-7e7f1c30dc98 admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:19:18.198893 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b674b8-7bb3-4dd6-92aa-7e7f1c30dc98 admin admin] PATH_INFO: `/v3/domains` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:19:18.199572 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b674b8-7bb3-4dd6-92aa-7e7f1c30dc98 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 12:19:18.199875 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b674b8-7bb3-4dd6-92aa-7e7f1c30dc98 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:19:18.201589 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b674b8-7bb3-4dd6-92aa-7e7f1c30dc98 admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:19:18.213988 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1212/2445] 10.222.0.46 () {58 vars in 1153 bytes} [Mon Oct 10 12:19:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 12:19:18.218754 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2028072-18ae-4631-a654-ad182b772e00 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:19:18.228171 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2028072-18ae-4631-a654-ad182b772e00 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:19:18.233528 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81458e2-9ae4-469c-a04e-576721e4e58a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:19:18.234164 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81458e2-9ae4-469c-a04e-576721e4e58a admin admin] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:19:18.234446 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81458e2-9ae4-469c-a04e-576721e4e58a admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:19:18.234772 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81458e2-9ae4-469c-a04e-576721e4e58a admin admin] PATH_INFO: `/v3/projects` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:19:18.235419 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81458e2-9ae4-469c-a04e-576721e4e58a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:19:18.237318 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81458e2-9ae4-469c-a04e-576721e4e58a admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:19:18.352791 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1234/2446] 10.222.0.46 () {60 vars in 1153 bytes} [Mon Oct 10 12:19:18 2022] POST /identity/v3/projects => generated 371 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 10 12:19:18.357616 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b674b8-7bb3-4dd6-92aa-7e7f1c30dc98 admin admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:19:18.367765 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b674b8-7bb3-4dd6-92aa-7e7f1c30dc98 admin admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:19:18.413378 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:19:18.427459 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b233a0-7518-4e15-959d-a46d8b2a08fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:19:18.428083 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b233a0-7518-4e15-959d-a46d8b2a08fa admin admin] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:19:18.428411 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b233a0-7518-4e15-959d-a46d8b2a08fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:19:18.428710 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b233a0-7518-4e15-959d-a46d8b2a08fa admin admin] PATH_INFO: `/v3/users` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:19:18.429343 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b233a0-7518-4e15-959d-a46d8b2a08fa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:19:18.431574 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b233a0-7518-4e15-959d-a46d8b2a08fa admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:19:18.539285 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1213/2447] 10.222.0.46 () {60 vars in 1144 bytes} [Mon Oct 10 12:19:18 2022] POST /identity/v3/users => generated 286 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 12:19:18.543852 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81458e2-9ae4-469c-a04e-576721e4e58a admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:19:18.554879 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81458e2-9ae4-469c-a04e-576721e4e58a admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:19:18.559509 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88416c72-621e-4465-86b4-56f477da67a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:19:18.560112 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88416c72-621e-4465-86b4-56f477da67a1 admin admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:19:18.560394 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88416c72-621e-4465-86b4-56f477da67a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:19:18.560667 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88416c72-621e-4465-86b4-56f477da67a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:19:18.561348 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88416c72-621e-4465-86b4-56f477da67a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:19:18.563056 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88416c72-621e-4465-86b4-56f477da67a1 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:19:18.573113 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1235/2448] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 10 12:19:18 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 12:19:18.577434 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b233a0-7518-4e15-959d-a46d8b2a08fa admin admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:19:18.590090 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b233a0-7518-4e15-959d-a46d8b2a08fa admin admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:19:18.594224 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c5fa6d-11a8-442d-a9ad-0f53f16d6ec2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:19:18.594869 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c5fa6d-11a8-442d-a9ad-0f53f16d6ec2 admin admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:19:18.595151 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c5fa6d-11a8-442d-a9ad-0f53f16d6ec2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:19:18.595428 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c5fa6d-11a8-442d-a9ad-0f53f16d6ec2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:19:18.596108 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c5fa6d-11a8-442d-a9ad-0f53f16d6ec2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:19:18.597694 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c5fa6d-11a8-442d-a9ad-0f53f16d6ec2 admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:19:18.609446 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1214/2449] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 10 12:19:18 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 12:19:18.613651 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88416c72-621e-4465-86b4-56f477da67a1 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:19:18.623620 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88416c72-621e-4465-86b4-56f477da67a1 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:19:18.628119 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8873430-6646-459a-92a6-4289a9e43fe2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:19:18.628769 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8873430-6646-459a-92a6-4289a9e43fe2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:19:18.629658 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8873430-6646-459a-92a6-4289a9e43fe2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:19:18.629963 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8873430-6646-459a-92a6-4289a9e43fe2 admin admin] PATH_INFO: `/v3/projects/c6ab1776de294ca1a23fc110b2f9af1d/users/8c796e1544354dbe9d1139c77b05f477/roles/b2cb2a6e818e47338ce071e6fedfc673` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:19:18.645347 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8873430-6646-459a-92a6-4289a9e43fe2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c6ab1776de294ca1a23fc110b2f9af1d, user_id=8c796e1544354dbe9d1139c77b05f477, role_id=b2cb2a6e818e47338ce071e6fedfc673)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:19:18.648066 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8873430-6646-459a-92a6-4289a9e43fe2 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:19:18.725745 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1236/2450] 10.222.0.46 () {60 vars in 1483 bytes} [Mon Oct 10 12:19:18 2022] PUT /identity/v3/projects/c6ab1776de294ca1a23fc110b2f9af1d/users/8c796e1544354dbe9d1139c77b05f477/roles/b2cb2a6e818e47338ce071e6fedfc673 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:19:18.731018 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a89cbd7-006a-4b45-92f6-fc9a1f31dd8a None None] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:19:18.731309 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a89cbd7-006a-4b45-92f6-fc9a1f31dd8a None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:19:18.731597 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a89cbd7-006a-4b45-92f6-fc9a1f31dd8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:19:18.751719 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.auth.core [None req-6a89cbd7-006a-4b45-92f6-fc9a1f31dd8a None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:19:18.752930 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-6a89cbd7-006a-4b45-92f6-fc9a1f31dd8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:19:18.810605 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1215/2451] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:19:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:19:18.820629 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8873430-6646-459a-92a6-4289a9e43fe2 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:19:18.822425 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-e8873430-6646-459a-92a6-4289a9e43fe2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:19:18.878987 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8873430-6646-459a-92a6-4289a9e43fe2 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:19:18.882571 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0816592-d989-47c6-beb5-f1dbec7ce55e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58b1c7c85302421cba823ace4201861f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad3772f7b6a47c28b7ba37a9316a3c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:19:18.883199 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0816592-d989-47c6-beb5-f1dbec7ce55e service neutron] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:19:18.883473 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0816592-d989-47c6-beb5-f1dbec7ce55e service neutron] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:19:18.883742 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0816592-d989-47c6-beb5-f1dbec7ce55e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:19:18.893237 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0816592-d989-47c6-beb5-f1dbec7ce55e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:19:18.894646 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0816592-d989-47c6-beb5-f1dbec7ce55e service neutron] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:19:18.903970 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1237/2452] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 10 12:19:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:19:19.801881 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a89cbd7-006a-4b45-92f6-fc9a1f31dd8a None None] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:19:19.803553 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-6a89cbd7-006a-4b45-92f6-fc9a1f31dd8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:19:19.882130 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a89cbd7-006a-4b45-92f6-fc9a1f31dd8a None None] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:19:19.888135 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8a4954-3414-4ec1-9480-9df38f73b798 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f130bbb69e546ce99b4e14de81d7e54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad3772f7b6a47c28b7ba37a9316a3c1', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:19:19.888744 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8a4954-3414-4ec1-9480-9df38f73b798 service nova] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:19:19.889023 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8a4954-3414-4ec1-9480-9df38f73b798 service nova] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:19:19.889299 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8a4954-3414-4ec1-9480-9df38f73b798 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:19:19.891224 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-6d8a4954-3414-4ec1-9480-9df38f73b798 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:19:19.986556 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:19:20.001780 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8a4954-3414-4ec1-9480-9df38f73b798 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:19:20.001780 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8a4954-3414-4ec1-9480-9df38f73b798 service nova] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:19:20.021198 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1216/2453] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 10 12:19:19 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 220 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:19:23.405296 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0816592-d989-47c6-beb5-f1dbec7ce55e service neutron] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:19:23.407412 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-b0816592-d989-47c6-beb5-f1dbec7ce55e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:19:23.503219 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:19:23.518173 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0816592-d989-47c6-beb5-f1dbec7ce55e service neutron] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:19:23.522379 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4ad005-b3d3-4290-9ed6-c97cf33e3693 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '124f4aa8ab244ca2aadf4f2614df51e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad3772f7b6a47c28b7ba37a9316a3c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:19:23.523018 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4ad005-b3d3-4290-9ed6-c97cf33e3693 service glance] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:19:23.523300 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4ad005-b3d3-4290-9ed6-c97cf33e3693 service glance] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:19:23.523565 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4ad005-b3d3-4290-9ed6-c97cf33e3693 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:19:23.525282 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-cc4ad005-b3d3-4290-9ed6-c97cf33e3693 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:19:23.539391 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4ad005-b3d3-4290-9ed6-c97cf33e3693 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:19:23.540832 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4ad005-b3d3-4290-9ed6-c97cf33e3693 service glance] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:19:23.552687 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1238/2454] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 10 12:19:23 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:19:23.920011 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8a4954-3414-4ec1-9480-9df38f73b798 service nova] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:19:23.929913 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8a4954-3414-4ec1-9480-9df38f73b798 service nova] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:19:23.935613 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a8b54d-1e53-4be2-873c-f942c49499fb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58b1c7c85302421cba823ace4201861f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad3772f7b6a47c28b7ba37a9316a3c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:19:23.936234 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a8b54d-1e53-4be2-873c-f942c49499fb service neutron] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:19:23.936712 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a8b54d-1e53-4be2-873c-f942c49499fb service neutron] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:19:23.936989 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a8b54d-1e53-4be2-873c-f942c49499fb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:19:23.939062 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-32a8b54d-1e53-4be2-873c-f942c49499fb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:19:23.949874 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a8b54d-1e53-4be2-873c-f942c49499fb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:19:23.951998 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a8b54d-1e53-4be2-873c-f942c49499fb service neutron] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:19:23.960565 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1217/2455] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 10 12:19:23 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:19:24.944540 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938ac68c-0aa4-4ddd-bb95-e1710d48a383 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:19:24.945079 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938ac68c-0aa4-4ddd-bb95-e1710d48a383 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:19:24.945079 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938ac68c-0aa4-4ddd-bb95-e1710d48a383 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:19:24.998150 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-938ac68c-0aa4-4ddd-bb95-e1710d48a383 None None] MFA Rules not processed for user `8c796e1544354dbe9d1139c77b05f477`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:19:25.000558 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-938ac68c-0aa4-4ddd-bb95-e1710d48a383 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:19:25.091682 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1239/2456] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:19:24 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:19:25.096433 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a8b54d-1e53-4be2-873c-f942c49499fb service neutron] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:19:25.113848 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a8b54d-1e53-4be2-873c-f942c49499fb service neutron] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:19:25.123924 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2ec9f6-8338-4d3a-8b99-39d5848efcb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:19:25.124759 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2ec9f6-8338-4d3a-8b99-39d5848efcb9 admin admin] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:19:25.125203 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2ec9f6-8338-4d3a-8b99-39d5848efcb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:19:25.125801 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2ec9f6-8338-4d3a-8b99-39d5848efcb9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:19:25.126459 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2ec9f6-8338-4d3a-8b99-39d5848efcb9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:19:25.128251 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938ac68c-0aa4-4ddd-bb95-e1710d48a383 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:19:25.128715 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2ec9f6-8338-4d3a-8b99-39d5848efcb9 admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:19:25.141234 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938ac68c-0aa4-4ddd-bb95-e1710d48a383 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:19:25.145950 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ef6947-d6b0-45c6-ad1d-6eaf95b5474b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58b1c7c85302421cba823ace4201861f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad3772f7b6a47c28b7ba37a9316a3c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:19:25.146752 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ef6947-d6b0-45c6-ad1d-6eaf95b5474b service neutron] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:19:25.149960 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ef6947-d6b0-45c6-ad1d-6eaf95b5474b service neutron] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:19:25.149960 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ef6947-d6b0-45c6-ad1d-6eaf95b5474b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:19:25.154639 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-79ef6947-d6b0-45c6-ad1d-6eaf95b5474b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:19:25.173925 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ef6947-d6b0-45c6-ad1d-6eaf95b5474b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:19:25.175714 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ef6947-d6b0-45c6-ad1d-6eaf95b5474b service neutron] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:19:25.189436 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1240/2457] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 10 12:19:25 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 10 12:19:25.223721 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1218/2458] 10.222.0.46 () {60 vars in 1153 bytes} [Mon Oct 10 12:19:25 2022] POST /identity/v3/projects => generated 373 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 12:19:25.224714 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ef6947-d6b0-45c6-ad1d-6eaf95b5474b service neutron] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:19:25.238125 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ef6947-d6b0-45c6-ad1d-6eaf95b5474b service neutron] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:19:25.319781 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93847079-e91a-40e4-9faf-208d54663098 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:19:25.320746 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93847079-e91a-40e4-9faf-208d54663098 admin admin] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:19:25.321187 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93847079-e91a-40e4-9faf-208d54663098 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:19:25.322239 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93847079-e91a-40e4-9faf-208d54663098 admin admin] PATH_INFO: `/v3/users` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:19:25.322790 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93847079-e91a-40e4-9faf-208d54663098 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:19:25.326373 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93847079-e91a-40e4-9faf-208d54663098 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:19:25.430426 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1241/2459] 10.222.0.46 () {60 vars in 1144 bytes} [Mon Oct 10 12:19:25 2022] POST /identity/v3/users => generated 293 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 12:19:25.434102 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2ec9f6-8338-4d3a-8b99-39d5848efcb9 admin admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:19:25.445734 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2ec9f6-8338-4d3a-8b99-39d5848efcb9 admin admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:19:25.450697 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11beb0c-1111-4105-a130-53e24e3b1de4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:19:25.451369 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11beb0c-1111-4105-a130-53e24e3b1de4 admin admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:19:25.451838 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11beb0c-1111-4105-a130-53e24e3b1de4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:19:25.452681 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11beb0c-1111-4105-a130-53e24e3b1de4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:19:25.452805 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11beb0c-1111-4105-a130-53e24e3b1de4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:19:25.454420 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11beb0c-1111-4105-a130-53e24e3b1de4 admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:19:25.468356 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1219/2460] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 10 12:19:25 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 12:19:25.472195 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93847079-e91a-40e4-9faf-208d54663098 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:19:25.483951 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93847079-e91a-40e4-9faf-208d54663098 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:19:25.489165 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7486e4c-fe09-4da0-a624-620b4c055a44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:19:25.489861 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7486e4c-fe09-4da0-a624-620b4c055a44 admin admin] REQUEST_METHOD: `PUT` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:19:25.490147 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7486e4c-fe09-4da0-a624-620b4c055a44 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:19:25.490431 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7486e4c-fe09-4da0-a624-620b4c055a44 admin admin] PATH_INFO: `/v3/projects/a198397258e44f2fa172d6b5289124de/users/32e9c2c51e664bb6acb8ce1693170385/roles/0d822a0674b6495fac623877a3b38025` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:19:25.508446 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7486e4c-fe09-4da0-a624-620b4c055a44 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a198397258e44f2fa172d6b5289124de, user_id=32e9c2c51e664bb6acb8ce1693170385, role_id=0d822a0674b6495fac623877a3b38025)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:19:25.511160 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7486e4c-fe09-4da0-a624-620b4c055a44 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:19:25.648410 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1242/2461] 10.222.0.46 () {60 vars in 1483 bytes} [Mon Oct 10 12:19:25 2022] PUT /identity/v3/projects/a198397258e44f2fa172d6b5289124de/users/32e9c2c51e664bb6acb8ce1693170385/roles/0d822a0674b6495fac623877a3b38025 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:19:25.652975 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11beb0c-1111-4105-a130-53e24e3b1de4 admin admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:19:25.664551 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11beb0c-1111-4105-a130-53e24e3b1de4 admin admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:19:25.725050 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a049ec57-5e84-401f-8521-e43cade5c90c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:19:25.725754 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a049ec57-5e84-401f-8521-e43cade5c90c admin admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:19:25.725928 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a049ec57-5e84-401f-8521-e43cade5c90c admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:19:25.726299 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a049ec57-5e84-401f-8521-e43cade5c90c admin admin] PATH_INFO: `/v3/roles` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:19:25.726867 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a049ec57-5e84-401f-8521-e43cade5c90c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:19:25.728505 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a049ec57-5e84-401f-8521-e43cade5c90c admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:19:25.740122 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1220/2462] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 10 12:19:25 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 12:19:25.744545 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7486e4c-fe09-4da0-a624-620b4c055a44 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:19:25.759648 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7486e4c-fe09-4da0-a624-620b4c055a44 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:19:25.764220 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df99e7c-4a76-4410-9c10-e4d5dd0e3a93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:19:25.764965 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df99e7c-4a76-4410-9c10-e4d5dd0e3a93 admin admin] REQUEST_METHOD: `PUT` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:19:25.765337 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df99e7c-4a76-4410-9c10-e4d5dd0e3a93 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:19:25.765797 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df99e7c-4a76-4410-9c10-e4d5dd0e3a93 admin admin] PATH_INFO: `/v3/projects/a198397258e44f2fa172d6b5289124de/users/32e9c2c51e664bb6acb8ce1693170385/roles/0d822a0674b6495fac623877a3b38025` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:19:25.767976 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df99e7c-4a76-4410-9c10-e4d5dd0e3a93 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a198397258e44f2fa172d6b5289124de, user_id=32e9c2c51e664bb6acb8ce1693170385, role_id=0d822a0674b6495fac623877a3b38025)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:19:25.770761 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df99e7c-4a76-4410-9c10-e4d5dd0e3a93 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:19:25.783751 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1243/2463] 10.222.0.46 () {60 vars in 1483 bytes} [Mon Oct 10 12:19:25 2022] PUT /identity/v3/projects/a198397258e44f2fa172d6b5289124de/users/32e9c2c51e664bb6acb8ce1693170385/roles/0d822a0674b6495fac623877a3b38025 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:19:32.340366 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddc095a-9c50-4a52-9dac-03fdc9e8a9e8 None None] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:19:32.341415 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddc095a-9c50-4a52-9dac-03fdc9e8a9e8 None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:19:32.342130 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddc095a-9c50-4a52-9dac-03fdc9e8a9e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:19:32.375172 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.auth.core [None req-6ddc095a-9c50-4a52-9dac-03fdc9e8a9e8 None None] MFA Rules not processed for user `32e9c2c51e664bb6acb8ce1693170385`. Rule list: `[]` (Enabled: `True`). {{(pid=81676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:19:32.376512 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-6ddc095a-9c50-4a52-9dac-03fdc9e8a9e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:19:32.442100 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec43d7d-8373-4e0f-844f-de7640ed0a1f None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:19:32.442639 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec43d7d-8373-4e0f-844f-de7640ed0a1f None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:19:32.442639 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec43d7d-8373-4e0f-844f-de7640ed0a1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:19:32.463720 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1221/2464] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:19:32 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:19:32.471044 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-1ec43d7d-8373-4e0f-844f-de7640ed0a1f None None] MFA Rules not processed for user `d6430593eecf45d8b891c3b55de4a944`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:19:32.472302 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-1ec43d7d-8373-4e0f-844f-de7640ed0a1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:19:32.478272 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:19:32.480082 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ddc095a-9c50-4a52-9dac-03fdc9e8a9e8 None None] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:19:32.499325 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:19:32.565617 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1244/2465] 10.222.0.46 () {58 vars in 1009 bytes} [Mon Oct 10 12:19:32 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 10 12:19:32.575183 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec43d7d-8373-4e0f-844f-de7640ed0a1f None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:19:32.577024 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-1ec43d7d-8373-4e0f-844f-de7640ed0a1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:19:32.591867 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ddc095a-9c50-4a52-9dac-03fdc9e8a9e8 None None] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:19:32.598505 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0971c4ba-5218-400a-85b1-b299fb7ba9ca service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f130bbb69e546ce99b4e14de81d7e54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad3772f7b6a47c28b7ba37a9316a3c1', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:19:32.599106 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0971c4ba-5218-400a-85b1-b299fb7ba9ca service nova] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:19:32.599382 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0971c4ba-5218-400a-85b1-b299fb7ba9ca service nova] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:19:32.599653 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0971c4ba-5218-400a-85b1-b299fb7ba9ca service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:19:32.673250 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec43d7d-8373-4e0f-844f-de7640ed0a1f None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:19:32.683197 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a0f4f8-c801-434f-bf2a-4b5ed6f0421a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15ef3d53ea484237adb0b3d64fdc5480', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad3772f7b6a47c28b7ba37a9316a3c1', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:19:32.685873 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a0f4f8-c801-434f-bf2a-4b5ed6f0421a service swift] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:19:32.686154 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a0f4f8-c801-434f-bf2a-4b5ed6f0421a service swift] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:19:32.686429 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a0f4f8-c801-434f-bf2a-4b5ed6f0421a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:19:32.687705 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0971c4ba-5218-400a-85b1-b299fb7ba9ca service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:19:32.689284 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0971c4ba-5218-400a-85b1-b299fb7ba9ca service nova] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:19:32.696912 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a0f4f8-c801-434f-bf2a-4b5ed6f0421a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:19:32.698537 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a0f4f8-c801-434f-bf2a-4b5ed6f0421a service swift] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:19:32.702870 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1222/2466] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 10 12:19:32 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 225 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:19:32.715213 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1245/2467] 10.222.0.46 () {60 vars in 1384 bytes} [Mon Oct 10 12:19:32 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 10 12:19:36.112125 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf44320-aa0b-4d40-b3b5-02c91c176d6c None None] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:19:36.118045 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf44320-aa0b-4d40-b3b5-02c91c176d6c None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:19:36.118697 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf44320-aa0b-4d40-b3b5-02c91c176d6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:19:36.147672 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.auth.core [None req-ecf44320-aa0b-4d40-b3b5-02c91c176d6c None None] MFA Rules not processed for user `8c796e1544354dbe9d1139c77b05f477`. Rule list: `[]` (Enabled: `True`). {{(pid=81676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:19:36.147672 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-ecf44320-aa0b-4d40-b3b5-02c91c176d6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:19:36.163655 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1223/2468] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:19:36 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:19:42.704025 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=52 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:19:46.138816 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=54 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:19:55.053391 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4fd414-557b-4258-9bfc-5890b95ca0fa None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:19:55.053903 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4fd414-557b-4258-9bfc-5890b95ca0fa None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:19:55.054056 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4fd414-557b-4258-9bfc-5890b95ca0fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:19:55.074824 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-9d4fd414-557b-4258-9bfc-5890b95ca0fa None None] MFA Rules not processed for user `d6430593eecf45d8b891c3b55de4a944`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:19:55.076055 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-9d4fd414-557b-4258-9bfc-5890b95ca0fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:19:55.083738 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1246/2469] 10.222.0.46 () {58 vars in 967 bytes} [Mon Oct 10 12:19:55 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:19:55.097630 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf44320-aa0b-4d40-b3b5-02c91c176d6c None None] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:19:55.110593 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf44320-aa0b-4d40-b3b5-02c91c176d6c None None] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:19:55.115613 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a62e9c2-9883-4b0b-a985-51b30a04767d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15ef3d53ea484237adb0b3d64fdc5480', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad3772f7b6a47c28b7ba37a9316a3c1', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:19:55.116215 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a62e9c2-9883-4b0b-a985-51b30a04767d service swift] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:19:55.116496 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a62e9c2-9883-4b0b-a985-51b30a04767d service swift] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:19:55.116767 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a62e9c2-9883-4b0b-a985-51b30a04767d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:19:55.125710 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a62e9c2-9883-4b0b-a985-51b30a04767d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:19:55.127336 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a62e9c2-9883-4b0b-a985-51b30a04767d service swift] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:19:55.136121 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1224/2470] 10.222.0.46 () {60 vars in 1384 bytes} [Mon Oct 10 12:19:55 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 10 12:20:05.069379 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=3 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:20:05.132333 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=6 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:20:11.180484 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4fd414-557b-4258-9bfc-5890b95ca0fa None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:20:11.181886 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-9d4fd414-557b-4258-9bfc-5890b95ca0fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:20:11.197693 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4fd414-557b-4258-9bfc-5890b95ca0fa None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:20:11.203185 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f5328b-273d-46ad-8765-c826709a800f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f130bbb69e546ce99b4e14de81d7e54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad3772f7b6a47c28b7ba37a9316a3c1', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:20:11.203800 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f5328b-273d-46ad-8765-c826709a800f service nova] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:20:11.204082 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f5328b-273d-46ad-8765-c826709a800f service nova] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:20:11.204357 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f5328b-273d-46ad-8765-c826709a800f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:20:11.206149 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-e8f5328b-273d-46ad-8765-c826709a800f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:20:11.218949 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f5328b-273d-46ad-8765-c826709a800f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:20:11.220630 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f5328b-273d-46ad-8765-c826709a800f service nova] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:20:11.232696 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1247/2471] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 10 12:20:11 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 10 12:20:16.189407 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986bc70d-e974-46a0-b6a8-e98a55e08c26 None None] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:20:16.189972 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986bc70d-e974-46a0-b6a8-e98a55e08c26 None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:20:16.189972 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986bc70d-e974-46a0-b6a8-e98a55e08c26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:20:16.239628 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.auth.core [None req-986bc70d-e974-46a0-b6a8-e98a55e08c26 None None] MFA Rules not processed for user `21c537b0fefe4fb1b51483ac0e3cf2d1`. Rule list: `[]` (Enabled: `True`). {{(pid=81676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:20:16.240838 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-986bc70d-e974-46a0-b6a8-e98a55e08c26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:20:16.306890 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1225/2472] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:20:16 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:20:21.227885 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=6 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:20:21.951274 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f5328b-273d-46ad-8765-c826709a800f service nova] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:20:22.006998 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f5328b-273d-46ad-8765-c826709a800f service nova] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:20:22.010738 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0b3487-55ff-41ca-b49c-acea6ed70b22 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58b1c7c85302421cba823ace4201861f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad3772f7b6a47c28b7ba37a9316a3c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:20:22.011360 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0b3487-55ff-41ca-b49c-acea6ed70b22 service neutron] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:20:22.011647 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0b3487-55ff-41ca-b49c-acea6ed70b22 service neutron] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:20:22.011940 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0b3487-55ff-41ca-b49c-acea6ed70b22 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:20:22.013709 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-4e0b3487-55ff-41ca-b49c-acea6ed70b22 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:20:22.084884 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0b3487-55ff-41ca-b49c-acea6ed70b22 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:20:22.086320 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0b3487-55ff-41ca-b49c-acea6ed70b22 service neutron] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:20:22.097192 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1248/2473] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 10 12:20:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:20:25.695781 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986bc70d-e974-46a0-b6a8-e98a55e08c26 None None] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:20:25.697569 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-986bc70d-e974-46a0-b6a8-e98a55e08c26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:20:25.773127 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986bc70d-e974-46a0-b6a8-e98a55e08c26 None None] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:20:25.778181 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c3407f-5499-4e20-b27a-35675673a4b6 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e8f9bb1776d47169ef7febb57c954cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad3772f7b6a47c28b7ba37a9316a3c1', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:20:25.778772 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c3407f-5499-4e20-b27a-35675673a4b6 service placement] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:20:25.779056 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c3407f-5499-4e20-b27a-35675673a4b6 service placement] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:20:25.779330 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c3407f-5499-4e20-b27a-35675673a4b6 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:20:25.781040 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-16c3407f-5499-4e20-b27a-35675673a4b6 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:20:25.793114 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c3407f-5499-4e20-b27a-35675673a4b6 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:20:25.794810 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c3407f-5499-4e20-b27a-35675673a4b6 service placement] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:20:25.807352 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0b3487-55ff-41ca-b49c-acea6ed70b22 service neutron] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:20:25.808938 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-4e0b3487-55ff-41ca-b49c-acea6ed70b22 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:20:25.810181 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1226/2474] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 10 12:20:25 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:20:25.829839 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0b3487-55ff-41ca-b49c-acea6ed70b22 service neutron] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:20:25.834658 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3def6eb4-28b5-4c30-b1db-0ea3bc2d217f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:20:25.835328 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3def6eb4-28b5-4c30-b1db-0ea3bc2d217f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:20:25.835612 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3def6eb4-28b5-4c30-b1db-0ea3bc2d217f admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:20:25.835891 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3def6eb4-28b5-4c30-b1db-0ea3bc2d217f admin admin] PATH_INFO: `/v3/users/b334b2401e14419284cce2ad61442d4a` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:20:25.837325 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3def6eb4-28b5-4c30-b1db-0ea3bc2d217f admin admin] RBAC: Authorizing `identity:delete_user(user_id=b334b2401e14419284cce2ad61442d4a)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:20:25.839625 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3def6eb4-28b5-4c30-b1db-0ea3bc2d217f admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:20:26.025969 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-3def6eb4-28b5-4c30-b1db-0ea3bc2d217f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b334b2401e14419284cce2ad61442d4a'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:20:26.033513 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-3def6eb4-28b5-4c30-b1db-0ea3bc2d217f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b334b2401e14419284cce2ad61442d4a'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:20:26.034086 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-3def6eb4-28b5-4c30-b1db-0ea3bc2d217f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b334b2401e14419284cce2ad61442d4a'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:20:26.108982 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-3def6eb4-28b5-4c30-b1db-0ea3bc2d217f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b334b2401e14419284cce2ad61442d4a'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:20:26.122042 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-3def6eb4-28b5-4c30-b1db-0ea3bc2d217f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b334b2401e14419284cce2ad61442d4a'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:20:26.128164 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1249/2475] 10.222.0.46 () {58 vars in 1224 bytes} [Mon Oct 10 12:20:25 2022] DELETE /identity/v3/users/b334b2401e14419284cce2ad61442d4a => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:20:26.356694 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c3407f-5499-4e20-b27a-35675673a4b6 service placement] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:20:26.369111 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c3407f-5499-4e20-b27a-35675673a4b6 service placement] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:20:26.428340 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f35e7e0-019c-47b5-9551-9ef82780a1aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:20:26.428973 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f35e7e0-019c-47b5-9551-9ef82780a1aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:20:26.429261 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f35e7e0-019c-47b5-9551-9ef82780a1aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:20:26.429608 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f35e7e0-019c-47b5-9551-9ef82780a1aa admin admin] PATH_INFO: `/v3/projects/98437356d32f4230bce856ce6ddae66a` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:20:26.430771 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f35e7e0-019c-47b5-9551-9ef82780a1aa admin admin] RBAC: Authorizing `identity:delete_project(project_id=98437356d32f4230bce856ce6ddae66a)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:20:26.432614 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f35e7e0-019c-47b5-9551-9ef82780a1aa admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:20:26.435236 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:20:26.546917 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-1f35e7e0-019c-47b5-9551-9ef82780a1aa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '98437356d32f4230bce856ce6ddae66a', 'request_id': 'req-1f35e7e0-019c-47b5-9551-9ef82780a1aa'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:20:26.552943 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-1f35e7e0-019c-47b5-9551-9ef82780a1aa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '98437356d32f4230bce856ce6ddae66a', 'request_id': 'req-1f35e7e0-019c-47b5-9551-9ef82780a1aa'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:20:26.559007 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-1f35e7e0-019c-47b5-9551-9ef82780a1aa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '98437356d32f4230bce856ce6ddae66a', 'request_id': 'req-1f35e7e0-019c-47b5-9551-9ef82780a1aa'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:20:26.666733 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-1f35e7e0-019c-47b5-9551-9ef82780a1aa admin admin] The token cache is being invalidate because project 98437356d32f4230bce856ce6ddae66a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81676) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 10 12:20:26.667030 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-1f35e7e0-019c-47b5-9551-9ef82780a1aa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:20:26.667660 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-1f35e7e0-019c-47b5-9551-9ef82780a1aa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:20:26.669160 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1227/2476] 10.222.0.46 () {58 vars in 1233 bytes} [Mon Oct 10 12:20:26 2022] DELETE /identity/v3/projects/98437356d32f4230bce856ce6ddae66a => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:20:26.674116 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29b2496-d5e7-470e-9191-694e232a3e9d None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:20:26.674409 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29b2496-d5e7-470e-9191-694e232a3e9d None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:20:26.674717 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29b2496-d5e7-470e-9191-694e232a3e9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:20:26.698140 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-e29b2496-d5e7-470e-9191-694e232a3e9d None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:20:26.700215 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-e29b2496-d5e7-470e-9191-694e232a3e9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:20:26.719636 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:20:26.762892 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1250/2477] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:20:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:20:26.770831 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75332265-f671-4f61-9276-a7081ebd72db None None] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:20:26.771312 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75332265-f671-4f61-9276-a7081ebd72db None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:20:26.771410 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75332265-f671-4f61-9276-a7081ebd72db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:20:26.791886 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.auth.core [None req-75332265-f671-4f61-9276-a7081ebd72db None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:20:26.793153 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-75332265-f671-4f61-9276-a7081ebd72db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:20:26.801972 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1228/2478] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:20:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:20:26.806224 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29b2496-d5e7-470e-9191-694e232a3e9d None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:20:26.816288 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29b2496-d5e7-470e-9191-694e232a3e9d None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:20:26.821686 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b244213c-34f3-4091-8a88-a283c9c6ae9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:20:26.822319 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b244213c-34f3-4091-8a88-a283c9c6ae9c admin admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:20:26.822594 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b244213c-34f3-4091-8a88-a283c9c6ae9c admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:20:26.822897 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b244213c-34f3-4091-8a88-a283c9c6ae9c admin admin] PATH_INFO: `/v3/domains` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:20:26.823569 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b244213c-34f3-4091-8a88-a283c9c6ae9c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 12:20:26.823872 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b244213c-34f3-4091-8a88-a283c9c6ae9c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:20:26.825453 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b244213c-34f3-4091-8a88-a283c9c6ae9c admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:20:26.837058 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1251/2479] 10.222.0.46 () {58 vars in 1153 bytes} [Mon Oct 10 12:20:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 12:20:26.841679 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75332265-f671-4f61-9276-a7081ebd72db None None] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:20:26.851939 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75332265-f671-4f61-9276-a7081ebd72db None None] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:20:26.860184 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a50a74a-aa89-4a4a-9e22-fa962baad7e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:20:26.861238 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a50a74a-aa89-4a4a-9e22-fa962baad7e8 admin admin] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:20:26.861526 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a50a74a-aa89-4a4a-9e22-fa962baad7e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:20:26.861806 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a50a74a-aa89-4a4a-9e22-fa962baad7e8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:20:26.862471 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a50a74a-aa89-4a4a-9e22-fa962baad7e8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:20:26.865031 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a50a74a-aa89-4a4a-9e22-fa962baad7e8 admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:20:26.950502 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1229/2480] 10.222.0.46 () {60 vars in 1153 bytes} [Mon Oct 10 12:20:26 2022] POST /identity/v3/projects => generated 379 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 10 12:20:26.954902 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b244213c-34f3-4091-8a88-a283c9c6ae9c admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:20:26.965523 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b244213c-34f3-4091-8a88-a283c9c6ae9c admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:20:27.024339 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0863fb0-4522-46fb-9df8-b035ddd9ae83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:20:27.024951 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0863fb0-4522-46fb-9df8-b035ddd9ae83 admin admin] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:20:27.025401 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0863fb0-4522-46fb-9df8-b035ddd9ae83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:20:27.025539 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0863fb0-4522-46fb-9df8-b035ddd9ae83 admin admin] PATH_INFO: `/v3/users` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:20:27.026141 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0863fb0-4522-46fb-9df8-b035ddd9ae83 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:20:27.028103 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0863fb0-4522-46fb-9df8-b035ddd9ae83 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:20:27.109804 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1252/2481] 10.222.0.46 () {60 vars in 1144 bytes} [Mon Oct 10 12:20:26 2022] POST /identity/v3/users => generated 290 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 12:20:27.113943 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a50a74a-aa89-4a4a-9e22-fa962baad7e8 admin admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:20:27.124565 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a50a74a-aa89-4a4a-9e22-fa962baad7e8 admin admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:20:27.129514 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ad95f1-7bc7-46a7-80d5-b5270f62974d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:20:27.130144 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ad95f1-7bc7-46a7-80d5-b5270f62974d admin admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:20:27.130459 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ad95f1-7bc7-46a7-80d5-b5270f62974d admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:20:27.130788 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ad95f1-7bc7-46a7-80d5-b5270f62974d admin admin] PATH_INFO: `/v3/roles` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:20:27.131458 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ad95f1-7bc7-46a7-80d5-b5270f62974d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:20:27.133052 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ad95f1-7bc7-46a7-80d5-b5270f62974d admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:20:27.143703 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1230/2482] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 10 12:20:27 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 12:20:27.148373 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0863fb0-4522-46fb-9df8-b035ddd9ae83 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:20:27.158986 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0863fb0-4522-46fb-9df8-b035ddd9ae83 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:20:27.164431 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b897f4-b82a-4f6b-a201-696f90c1f244 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:20:27.165039 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b897f4-b82a-4f6b-a201-696f90c1f244 admin admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:20:27.165318 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b897f4-b82a-4f6b-a201-696f90c1f244 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:20:27.165591 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b897f4-b82a-4f6b-a201-696f90c1f244 admin admin] PATH_INFO: `/v3/roles` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:20:27.166240 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b897f4-b82a-4f6b-a201-696f90c1f244 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:20:27.167994 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b897f4-b82a-4f6b-a201-696f90c1f244 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:20:27.179829 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1253/2483] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 10 12:20:27 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 12:20:27.184214 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ad95f1-7bc7-46a7-80d5-b5270f62974d admin admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:20:27.194172 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ad95f1-7bc7-46a7-80d5-b5270f62974d admin admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:20:27.198799 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0687c9-6cb1-4718-8a79-a4180058a364 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:20:27.199449 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0687c9-6cb1-4718-8a79-a4180058a364 admin admin] REQUEST_METHOD: `PUT` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:20:27.199721 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0687c9-6cb1-4718-8a79-a4180058a364 admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:20:27.199997 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0687c9-6cb1-4718-8a79-a4180058a364 admin admin] PATH_INFO: `/v3/projects/5ed011593c8544f4b2ae6a8760a947c3/users/6e1f08d3b99a431fac677758758f3e68/roles/b2cb2a6e818e47338ce071e6fedfc673` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:20:27.216091 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0687c9-6cb1-4718-8a79-a4180058a364 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5ed011593c8544f4b2ae6a8760a947c3, user_id=6e1f08d3b99a431fac677758758f3e68, role_id=b2cb2a6e818e47338ce071e6fedfc673)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:20:27.218751 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0687c9-6cb1-4718-8a79-a4180058a364 admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:20:27.286979 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1231/2484] 10.222.0.46 () {60 vars in 1483 bytes} [Mon Oct 10 12:20:27 2022] PUT /identity/v3/projects/5ed011593c8544f4b2ae6a8760a947c3/users/6e1f08d3b99a431fac677758758f3e68/roles/b2cb2a6e818e47338ce071e6fedfc673 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:20:27.297323 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a114962b-9a3c-42b4-aa89-0c8ea356712e None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:20:27.297612 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a114962b-9a3c-42b4-aa89-0c8ea356712e None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:20:27.297889 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a114962b-9a3c-42b4-aa89-0c8ea356712e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:20:27.327423 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-a114962b-9a3c-42b4-aa89-0c8ea356712e None None] MFA Rules not processed for user `6e1f08d3b99a431fac677758758f3e68`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:20:27.328641 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-a114962b-9a3c-42b4-aa89-0c8ea356712e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:20:27.392572 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1254/2485] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:20:27 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:20:27.397839 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0687c9-6cb1-4718-8a79-a4180058a364 admin admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:20:27.476041 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0687c9-6cb1-4718-8a79-a4180058a364 admin admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:20:27.488077 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e76b3f-bb2f-4724-b2b0-1083d0dfd675 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:20:27.488705 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e76b3f-bb2f-4724-b2b0-1083d0dfd675 admin admin] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:20:27.488976 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e76b3f-bb2f-4724-b2b0-1083d0dfd675 admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:20:27.489244 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e76b3f-bb2f-4724-b2b0-1083d0dfd675 admin admin] PATH_INFO: `/v3/projects` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:20:27.489889 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e76b3f-bb2f-4724-b2b0-1083d0dfd675 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:20:27.492353 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e76b3f-bb2f-4724-b2b0-1083d0dfd675 admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:20:27.568481 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1232/2486] 10.222.0.46 () {60 vars in 1153 bytes} [Mon Oct 10 12:20:27 2022] POST /identity/v3/projects => generated 377 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 10 12:20:27.572629 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a114962b-9a3c-42b4-aa89-0c8ea356712e None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:20:27.583975 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a114962b-9a3c-42b4-aa89-0c8ea356712e None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:20:27.597092 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:20:27.643015 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66568244-320a-4beb-aa36-571297239e6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:20:27.643733 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66568244-320a-4beb-aa36-571297239e6f admin admin] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:20:27.644208 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66568244-320a-4beb-aa36-571297239e6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:20:27.644484 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66568244-320a-4beb-aa36-571297239e6f admin admin] PATH_INFO: `/v3/users` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:20:27.645759 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66568244-320a-4beb-aa36-571297239e6f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:20:27.648265 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66568244-320a-4beb-aa36-571297239e6f admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:20:27.769295 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1255/2487] 10.222.0.46 () {60 vars in 1144 bytes} [Mon Oct 10 12:20:27 2022] POST /identity/v3/users => generated 295 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 12:20:27.774634 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e76b3f-bb2f-4724-b2b0-1083d0dfd675 admin admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:20:27.790423 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e76b3f-bb2f-4724-b2b0-1083d0dfd675 admin admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:20:27.795811 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53721ba-5887-4c75-b2d4-8f19bc9d3e73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:20:27.796410 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53721ba-5887-4c75-b2d4-8f19bc9d3e73 admin admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:20:27.803707 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53721ba-5887-4c75-b2d4-8f19bc9d3e73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:20:27.803707 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53721ba-5887-4c75-b2d4-8f19bc9d3e73 admin admin] PATH_INFO: `/v3/roles` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:20:27.803707 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53721ba-5887-4c75-b2d4-8f19bc9d3e73 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:20:27.803707 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53721ba-5887-4c75-b2d4-8f19bc9d3e73 admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:20:27.805979 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:20:27.811905 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1233/2488] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 10 12:20:27 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 10 12:20:27.817026 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66568244-320a-4beb-aa36-571297239e6f admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:20:27.828660 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66568244-320a-4beb-aa36-571297239e6f admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:20:27.834072 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bca5e8-dc50-4f77-b746-727a8f7447c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:20:27.834768 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bca5e8-dc50-4f77-b746-727a8f7447c5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:20:27.835054 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bca5e8-dc50-4f77-b746-727a8f7447c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:20:27.835334 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bca5e8-dc50-4f77-b746-727a8f7447c5 admin admin] PATH_INFO: `/v3/projects/5f2113978b41471cbd3bfe4299e85c68/users/c785c42c097c4746aae5114de47ce829/roles/0d822a0674b6495fac623877a3b38025` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:20:27.851977 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2bca5e8-dc50-4f77-b746-727a8f7447c5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5f2113978b41471cbd3bfe4299e85c68, user_id=c785c42c097c4746aae5114de47ce829, role_id=0d822a0674b6495fac623877a3b38025)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:20:27.854694 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2bca5e8-dc50-4f77-b746-727a8f7447c5 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:20:27.943606 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1256/2489] 10.222.0.46 () {60 vars in 1483 bytes} [Mon Oct 10 12:20:27 2022] PUT /identity/v3/projects/5f2113978b41471cbd3bfe4299e85c68/users/c785c42c097c4746aae5114de47ce829/roles/0d822a0674b6495fac623877a3b38025 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:20:27.947795 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53721ba-5887-4c75-b2d4-8f19bc9d3e73 admin admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:20:27.958834 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53721ba-5887-4c75-b2d4-8f19bc9d3e73 admin admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:20:28.016875 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b35ca0-3468-4653-8856-343cfdb13c24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:20:28.017486 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b35ca0-3468-4653-8856-343cfdb13c24 admin admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:20:28.017767 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b35ca0-3468-4653-8856-343cfdb13c24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:20:28.018043 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b35ca0-3468-4653-8856-343cfdb13c24 admin admin] PATH_INFO: `/v3/roles` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:20:28.018719 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b35ca0-3468-4653-8856-343cfdb13c24 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:20:28.020965 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b35ca0-3468-4653-8856-343cfdb13c24 admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:20:28.030939 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1234/2490] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 10 12:20:27 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 12:20:28.035366 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bca5e8-dc50-4f77-b746-727a8f7447c5 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:20:28.046114 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bca5e8-dc50-4f77-b746-727a8f7447c5 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:20:28.051125 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3852d9cd-44f7-4c0a-95c5-2797a07d2485 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:20:28.051778 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3852d9cd-44f7-4c0a-95c5-2797a07d2485 admin admin] REQUEST_METHOD: `PUT` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:20:28.052056 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3852d9cd-44f7-4c0a-95c5-2797a07d2485 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:20:28.052328 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3852d9cd-44f7-4c0a-95c5-2797a07d2485 admin admin] PATH_INFO: `/v3/projects/5f2113978b41471cbd3bfe4299e85c68/users/c785c42c097c4746aae5114de47ce829/roles/0d822a0674b6495fac623877a3b38025` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:20:28.054244 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3852d9cd-44f7-4c0a-95c5-2797a07d2485 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5f2113978b41471cbd3bfe4299e85c68, user_id=c785c42c097c4746aae5114de47ce829, role_id=0d822a0674b6495fac623877a3b38025)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:20:28.056894 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3852d9cd-44f7-4c0a-95c5-2797a07d2485 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:20:28.068715 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1257/2491] 10.222.0.46 () {60 vars in 1483 bytes} [Mon Oct 10 12:20:28 2022] PUT /identity/v3/projects/5f2113978b41471cbd3bfe4299e85c68/users/c785c42c097c4746aae5114de47ce829/roles/0d822a0674b6495fac623877a3b38025 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:20:28.080421 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c9ace8-f086-4a8a-9edc-cfd5ed7207f2 None None] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:20:28.080798 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c9ace8-f086-4a8a-9edc-cfd5ed7207f2 None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:20:28.081141 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c9ace8-f086-4a8a-9edc-cfd5ed7207f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:20:28.110553 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.auth.core [None req-84c9ace8-f086-4a8a-9edc-cfd5ed7207f2 None None] MFA Rules not processed for user `c785c42c097c4746aae5114de47ce829`. Rule list: `[]` (Enabled: `True`). {{(pid=81676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:20:28.111843 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-84c9ace8-f086-4a8a-9edc-cfd5ed7207f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:20:28.182227 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1235/2492] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:20:28 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:20:28.195911 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3852d9cd-44f7-4c0a-95c5-2797a07d2485 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:20:28.197734 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-3852d9cd-44f7-4c0a-95c5-2797a07d2485 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:20:28.262162 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3852d9cd-44f7-4c0a-95c5-2797a07d2485 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:20:28.266609 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eaf1b58-81d8-428b-a837-175ac7732e06 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3ae093470484b93b23dcfae1657df9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad3772f7b6a47c28b7ba37a9316a3c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:20:28.271679 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eaf1b58-81d8-428b-a837-175ac7732e06 service cinder] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:20:28.271679 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eaf1b58-81d8-428b-a837-175ac7732e06 service cinder] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:20:28.271679 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eaf1b58-81d8-428b-a837-175ac7732e06 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:20:28.278634 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eaf1b58-81d8-428b-a837-175ac7732e06 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:20:28.280121 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eaf1b58-81d8-428b-a837-175ac7732e06 service cinder] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:20:28.291031 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1258/2493] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 10 12:20:28 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:20:28.920236 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c9ace8-f086-4a8a-9edc-cfd5ed7207f2 None None] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:20:28.921723 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-84c9ace8-f086-4a8a-9edc-cfd5ed7207f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:20:28.980264 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c9ace8-f086-4a8a-9edc-cfd5ed7207f2 None None] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:20:28.983881 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2795b00a-7206-4998-b471-97352b155ec0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58b1c7c85302421cba823ace4201861f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad3772f7b6a47c28b7ba37a9316a3c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:20:28.984483 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2795b00a-7206-4998-b471-97352b155ec0 service neutron] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:20:28.984765 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2795b00a-7206-4998-b471-97352b155ec0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:20:28.985097 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2795b00a-7206-4998-b471-97352b155ec0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:20:28.986807 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-2795b00a-7206-4998-b471-97352b155ec0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:20:29.021508 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:20:29.050241 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2795b00a-7206-4998-b471-97352b155ec0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:20:29.052347 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2795b00a-7206-4998-b471-97352b155ec0 service neutron] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:20:29.062636 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1236/2494] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 10 12:20:28 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:20:29.256865 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eaf1b58-81d8-428b-a837-175ac7732e06 service cinder] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:20:29.266277 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eaf1b58-81d8-428b-a837-175ac7732e06 service cinder] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:20:29.271743 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1cd938-f583-421c-8d23-5b0dcbdea7e7 tempest-VolumeQuotaClassesTest-29457713 tempest-VolumeQuotaClassesTest-29457713-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c785c42c097c4746aae5114de47ce829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f2113978b41471cbd3bfe4299e85c68', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:20:29.272405 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1cd938-f583-421c-8d23-5b0dcbdea7e7 tempest-VolumeQuotaClassesTest-29457713 tempest-VolumeQuotaClassesTest-29457713-project-admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:20:29.272756 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1cd938-f583-421c-8d23-5b0dcbdea7e7 tempest-VolumeQuotaClassesTest-29457713 tempest-VolumeQuotaClassesTest-29457713-project-admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:20:29.273087 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1cd938-f583-421c-8d23-5b0dcbdea7e7 tempest-VolumeQuotaClassesTest-29457713 tempest-VolumeQuotaClassesTest-29457713-project-admin] PATH_INFO: `/v3/domains` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:20:29.273749 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1cd938-f583-421c-8d23-5b0dcbdea7e7 tempest-VolumeQuotaClassesTest-29457713 tempest-VolumeQuotaClassesTest-29457713-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 12:20:29.274137 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1cd938-f583-421c-8d23-5b0dcbdea7e7 tempest-VolumeQuotaClassesTest-29457713 tempest-VolumeQuotaClassesTest-29457713-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:20:29.275815 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1cd938-f583-421c-8d23-5b0dcbdea7e7 tempest-VolumeQuotaClassesTest-29457713 tempest-VolumeQuotaClassesTest-29457713-project-admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:20:29.291313 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1259/2495] 10.222.0.46 () {58 vars in 1153 bytes} [Mon Oct 10 12:20:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 12:20:29.292692 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2795b00a-7206-4998-b471-97352b155ec0 service neutron] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:20:29.303382 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2795b00a-7206-4998-b471-97352b155ec0 service neutron] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:20:29.309798 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401146b5-43c1-436d-a233-dfd9ad6cb6a6 tempest-VolumeQuotaClassesTest-29457713 tempest-VolumeQuotaClassesTest-29457713-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c785c42c097c4746aae5114de47ce829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f2113978b41471cbd3bfe4299e85c68', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:20:29.309798 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401146b5-43c1-436d-a233-dfd9ad6cb6a6 tempest-VolumeQuotaClassesTest-29457713 tempest-VolumeQuotaClassesTest-29457713-project-admin] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:20:29.309798 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401146b5-43c1-436d-a233-dfd9ad6cb6a6 tempest-VolumeQuotaClassesTest-29457713 tempest-VolumeQuotaClassesTest-29457713-project-admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:20:29.310261 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401146b5-43c1-436d-a233-dfd9ad6cb6a6 tempest-VolumeQuotaClassesTest-29457713 tempest-VolumeQuotaClassesTest-29457713-project-admin] PATH_INFO: `/v3/projects` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:20:29.310261 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401146b5-43c1-436d-a233-dfd9ad6cb6a6 tempest-VolumeQuotaClassesTest-29457713 tempest-VolumeQuotaClassesTest-29457713-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:20:29.311903 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401146b5-43c1-436d-a233-dfd9ad6cb6a6 tempest-VolumeQuotaClassesTest-29457713 tempest-VolumeQuotaClassesTest-29457713-project-admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:20:29.422349 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1237/2496] 10.222.0.46 () {60 vars in 1153 bytes} [Mon Oct 10 12:20:29 2022] POST /identity/v3/projects => generated 353 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 12:20:29.426764 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1cd938-f583-421c-8d23-5b0dcbdea7e7 tempest-VolumeQuotaClassesTest-29457713 tempest-VolumeQuotaClassesTest-29457713-project-admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:20:29.436919 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1cd938-f583-421c-8d23-5b0dcbdea7e7 tempest-VolumeQuotaClassesTest-29457713 tempest-VolumeQuotaClassesTest-29457713-project-admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:20:29.500498 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e346dd5-d016-4d2e-9f35-660d6ea76f8a tempest-VolumeQuotaClassesTest-29457713 tempest-VolumeQuotaClassesTest-29457713-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c785c42c097c4746aae5114de47ce829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f2113978b41471cbd3bfe4299e85c68', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:20:29.504457 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e346dd5-d016-4d2e-9f35-660d6ea76f8a tempest-VolumeQuotaClassesTest-29457713 tempest-VolumeQuotaClassesTest-29457713-project-admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:20:29.504457 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e346dd5-d016-4d2e-9f35-660d6ea76f8a tempest-VolumeQuotaClassesTest-29457713 tempest-VolumeQuotaClassesTest-29457713-project-admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:20:29.504457 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e346dd5-d016-4d2e-9f35-660d6ea76f8a tempest-VolumeQuotaClassesTest-29457713 tempest-VolumeQuotaClassesTest-29457713-project-admin] PATH_INFO: `/v3/domains` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:20:29.504457 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e346dd5-d016-4d2e-9f35-660d6ea76f8a tempest-VolumeQuotaClassesTest-29457713 tempest-VolumeQuotaClassesTest-29457713-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 12:20:29.504457 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e346dd5-d016-4d2e-9f35-660d6ea76f8a tempest-VolumeQuotaClassesTest-29457713 tempest-VolumeQuotaClassesTest-29457713-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:20:29.504928 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e346dd5-d016-4d2e-9f35-660d6ea76f8a tempest-VolumeQuotaClassesTest-29457713 tempest-VolumeQuotaClassesTest-29457713-project-admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:20:29.505869 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:20:29.516885 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1260/2497] 10.222.0.46 () {58 vars in 1153 bytes} [Mon Oct 10 12:20:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 12:20:29.558543 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401146b5-43c1-436d-a233-dfd9ad6cb6a6 tempest-VolumeQuotaClassesTest-29457713 tempest-VolumeQuotaClassesTest-29457713-project-admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:20:29.569436 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401146b5-43c1-436d-a233-dfd9ad6cb6a6 tempest-VolumeQuotaClassesTest-29457713 tempest-VolumeQuotaClassesTest-29457713-project-admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:20:29.574302 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca58ce58-5aeb-4320-87fc-b15db14e7957 tempest-VolumeQuotaClassesTest-29457713 tempest-VolumeQuotaClassesTest-29457713-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c785c42c097c4746aae5114de47ce829', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f2113978b41471cbd3bfe4299e85c68', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:20:29.574998 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca58ce58-5aeb-4320-87fc-b15db14e7957 tempest-VolumeQuotaClassesTest-29457713 tempest-VolumeQuotaClassesTest-29457713-project-admin] REQUEST_METHOD: `DELETE` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:20:29.575315 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca58ce58-5aeb-4320-87fc-b15db14e7957 tempest-VolumeQuotaClassesTest-29457713 tempest-VolumeQuotaClassesTest-29457713-project-admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:20:29.575560 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca58ce58-5aeb-4320-87fc-b15db14e7957 tempest-VolumeQuotaClassesTest-29457713 tempest-VolumeQuotaClassesTest-29457713-project-admin] PATH_INFO: `/v3/projects/d52f0e38174d44f6baf7b4cb1b1e847f` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:20:29.577842 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca58ce58-5aeb-4320-87fc-b15db14e7957 tempest-VolumeQuotaClassesTest-29457713 tempest-VolumeQuotaClassesTest-29457713-project-admin] RBAC: Authorizing `identity:delete_project(project_id=d52f0e38174d44f6baf7b4cb1b1e847f)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:20:29.578692 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca58ce58-5aeb-4320-87fc-b15db14e7957 tempest-VolumeQuotaClassesTest-29457713 tempest-VolumeQuotaClassesTest-29457713-project-admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:20:29.727204 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-ca58ce58-5aeb-4320-87fc-b15db14e7957 tempest-VolumeQuotaClassesTest-29457713 tempest-VolumeQuotaClassesTest-29457713-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd52f0e38174d44f6baf7b4cb1b1e847f', 'request_id': 'req-ca58ce58-5aeb-4320-87fc-b15db14e7957'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:20:29.733221 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-ca58ce58-5aeb-4320-87fc-b15db14e7957 tempest-VolumeQuotaClassesTest-29457713 tempest-VolumeQuotaClassesTest-29457713-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd52f0e38174d44f6baf7b4cb1b1e847f', 'request_id': 'req-ca58ce58-5aeb-4320-87fc-b15db14e7957'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:20:29.740786 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-ca58ce58-5aeb-4320-87fc-b15db14e7957 tempest-VolumeQuotaClassesTest-29457713 tempest-VolumeQuotaClassesTest-29457713-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd52f0e38174d44f6baf7b4cb1b1e847f', 'request_id': 'req-ca58ce58-5aeb-4320-87fc-b15db14e7957'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:20:29.821502 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-ca58ce58-5aeb-4320-87fc-b15db14e7957 tempest-VolumeQuotaClassesTest-29457713 tempest-VolumeQuotaClassesTest-29457713-project-admin] The token cache is being invalidate because project d52f0e38174d44f6baf7b4cb1b1e847f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81676) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 10 12:20:29.821822 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-ca58ce58-5aeb-4320-87fc-b15db14e7957 tempest-VolumeQuotaClassesTest-29457713 tempest-VolumeQuotaClassesTest-29457713-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:20:29.822593 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-ca58ce58-5aeb-4320-87fc-b15db14e7957 tempest-VolumeQuotaClassesTest-29457713 tempest-VolumeQuotaClassesTest-29457713-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:20:29.824249 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1238/2498] 10.222.0.46 () {58 vars in 1233 bytes} [Mon Oct 10 12:20:29 2022] DELETE /identity/v3/projects/d52f0e38174d44f6baf7b4cb1b1e847f => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:20:30.444621 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e346dd5-d016-4d2e-9f35-660d6ea76f8a tempest-VolumeQuotaClassesTest-29457713 tempest-VolumeQuotaClassesTest-29457713-project-admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:20:30.446468 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-9e346dd5-d016-4d2e-9f35-660d6ea76f8a tempest-VolumeQuotaClassesTest-29457713 tempest-VolumeQuotaClassesTest-29457713-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:20:30.517497 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e346dd5-d016-4d2e-9f35-660d6ea76f8a tempest-VolumeQuotaClassesTest-29457713 tempest-VolumeQuotaClassesTest-29457713-project-admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:20:30.582597 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea76e80-4467-4b09-adad-577d146a2417 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:20:30.583272 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea76e80-4467-4b09-adad-577d146a2417 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:20:30.583559 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea76e80-4467-4b09-adad-577d146a2417 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:20:30.583834 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea76e80-4467-4b09-adad-577d146a2417 admin admin] PATH_INFO: `/v3/users/6e1f08d3b99a431fac677758758f3e68` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:20:30.584972 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea76e80-4467-4b09-adad-577d146a2417 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6e1f08d3b99a431fac677758758f3e68)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:20:30.587041 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea76e80-4467-4b09-adad-577d146a2417 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:20:30.753621 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-aea76e80-4467-4b09-adad-577d146a2417 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6e1f08d3b99a431fac677758758f3e68'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:20:30.761756 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-aea76e80-4467-4b09-adad-577d146a2417 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6e1f08d3b99a431fac677758758f3e68'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:20:30.762478 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-aea76e80-4467-4b09-adad-577d146a2417 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6e1f08d3b99a431fac677758758f3e68'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:20:30.840470 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-aea76e80-4467-4b09-adad-577d146a2417 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6e1f08d3b99a431fac677758758f3e68'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:20:30.857338 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-aea76e80-4467-4b09-adad-577d146a2417 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6e1f08d3b99a431fac677758758f3e68'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:20:30.863927 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1261/2499] 10.222.0.46 () {58 vars in 1224 bytes} [Mon Oct 10 12:20:30 2022] DELETE /identity/v3/users/6e1f08d3b99a431fac677758758f3e68 => generated 0 bytes in 420 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:20:30.868597 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439cdaa7-c023-45da-9f17-4a3a24480a0c None None] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:20:30.869673 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439cdaa7-c023-45da-9f17-4a3a24480a0c None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:20:30.869673 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439cdaa7-c023-45da-9f17-4a3a24480a0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:20:30.888793 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.auth.core [None req-439cdaa7-c023-45da-9f17-4a3a24480a0c None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:20:30.890015 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-439cdaa7-c023-45da-9f17-4a3a24480a0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:20:30.953357 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1239/2500] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:20:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:20:30.963519 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea76e80-4467-4b09-adad-577d146a2417 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:20:30.964979 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-aea76e80-4467-4b09-adad-577d146a2417 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:20:30.991844 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:20:31.029025 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea76e80-4467-4b09-adad-577d146a2417 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:20:31.032825 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1c6946-dbfd-4f60-93a1-b1ff801cf44d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58b1c7c85302421cba823ace4201861f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad3772f7b6a47c28b7ba37a9316a3c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:20:31.033443 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1c6946-dbfd-4f60-93a1-b1ff801cf44d service neutron] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:20:31.034853 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1c6946-dbfd-4f60-93a1-b1ff801cf44d service neutron] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:20:31.034853 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1c6946-dbfd-4f60-93a1-b1ff801cf44d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:20:31.045772 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1c6946-dbfd-4f60-93a1-b1ff801cf44d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:20:31.047275 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1c6946-dbfd-4f60-93a1-b1ff801cf44d service neutron] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:20:31.059031 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1262/2501] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 10 12:20:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:20:31.531846 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439cdaa7-c023-45da-9f17-4a3a24480a0c None None] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:20:31.534136 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-439cdaa7-c023-45da-9f17-4a3a24480a0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:20:31.554038 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439cdaa7-c023-45da-9f17-4a3a24480a0c None None] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:20:31.561046 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8f2114-6628-4fab-a2bb-ca4228c83964 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:20:31.562528 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8f2114-6628-4fab-a2bb-ca4228c83964 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:20:31.563237 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8f2114-6628-4fab-a2bb-ca4228c83964 admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:20:31.563797 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8f2114-6628-4fab-a2bb-ca4228c83964 admin admin] PATH_INFO: `/v3/projects/5ed011593c8544f4b2ae6a8760a947c3` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:20:31.565591 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8f2114-6628-4fab-a2bb-ca4228c83964 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5ed011593c8544f4b2ae6a8760a947c3)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:20:31.567972 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8f2114-6628-4fab-a2bb-ca4228c83964 admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:20:31.674641 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=89,keystone:INSERT=2,keystone:DELETE=9 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:20:31.691321 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-6b8f2114-6628-4fab-a2bb-ca4228c83964 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5ed011593c8544f4b2ae6a8760a947c3', 'request_id': 'req-6b8f2114-6628-4fab-a2bb-ca4228c83964'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:20:31.697169 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-6b8f2114-6628-4fab-a2bb-ca4228c83964 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5ed011593c8544f4b2ae6a8760a947c3', 'request_id': 'req-6b8f2114-6628-4fab-a2bb-ca4228c83964'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:20:31.703704 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-6b8f2114-6628-4fab-a2bb-ca4228c83964 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5ed011593c8544f4b2ae6a8760a947c3', 'request_id': 'req-6b8f2114-6628-4fab-a2bb-ca4228c83964'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:20:31.801258 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-6b8f2114-6628-4fab-a2bb-ca4228c83964 admin admin] The token cache is being invalidate because project 5ed011593c8544f4b2ae6a8760a947c3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81676) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 10 12:20:31.801563 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-6b8f2114-6628-4fab-a2bb-ca4228c83964 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:20:31.802359 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-6b8f2114-6628-4fab-a2bb-ca4228c83964 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:20:31.810391 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1c6946-dbfd-4f60-93a1-b1ff801cf44d service neutron] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:20:31.830257 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1240/2502] 10.222.0.46 () {58 vars in 1233 bytes} [Mon Oct 10 12:20:31 2022] DELETE /identity/v3/projects/5ed011593c8544f4b2ae6a8760a947c3 => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:20:31.835945 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1c6946-dbfd-4f60-93a1-b1ff801cf44d service neutron] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:20:31.923752 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8079e9f-279f-43a4-a1ee-d81e638b04e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:20:31.924466 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8079e9f-279f-43a4-a1ee-d81e638b04e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:20:31.924917 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8079e9f-279f-43a4-a1ee-d81e638b04e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:20:31.925194 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8079e9f-279f-43a4-a1ee-d81e638b04e5 admin admin] PATH_INFO: `/v3/users/c785c42c097c4746aae5114de47ce829` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:20:31.927495 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8079e9f-279f-43a4-a1ee-d81e638b04e5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c785c42c097c4746aae5114de47ce829)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:20:31.929438 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8079e9f-279f-43a4-a1ee-d81e638b04e5 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:20:32.158425 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-a8079e9f-279f-43a4-a1ee-d81e638b04e5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c785c42c097c4746aae5114de47ce829'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:20:32.166135 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-a8079e9f-279f-43a4-a1ee-d81e638b04e5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c785c42c097c4746aae5114de47ce829'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:20:32.166918 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-a8079e9f-279f-43a4-a1ee-d81e638b04e5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c785c42c097c4746aae5114de47ce829'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:20:32.263274 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-a8079e9f-279f-43a4-a1ee-d81e638b04e5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c785c42c097c4746aae5114de47ce829'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:20:32.279043 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-a8079e9f-279f-43a4-a1ee-d81e638b04e5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c785c42c097c4746aae5114de47ce829'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:20:32.286762 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1263/2503] 10.222.0.46 () {58 vars in 1224 bytes} [Mon Oct 10 12:20:31 2022] DELETE /identity/v3/users/c785c42c097c4746aae5114de47ce829 => generated 0 bytes in 479 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:20:32.745058 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8f2114-6628-4fab-a2bb-ca4228c83964 admin admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:20:32.746628 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-6b8f2114-6628-4fab-a2bb-ca4228c83964 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:20:32.759674 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8f2114-6628-4fab-a2bb-ca4228c83964 admin admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:20:32.816294 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca47fe6-be01-40db-94de-a835a32b052a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:20:32.816858 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca47fe6-be01-40db-94de-a835a32b052a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:20:32.817153 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca47fe6-be01-40db-94de-a835a32b052a admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:20:32.817422 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca47fe6-be01-40db-94de-a835a32b052a admin admin] PATH_INFO: `/v3/projects/5f2113978b41471cbd3bfe4299e85c68` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:20:32.818548 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca47fe6-be01-40db-94de-a835a32b052a admin admin] RBAC: Authorizing `identity:delete_project(project_id=5f2113978b41471cbd3bfe4299e85c68)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:20:32.820479 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca47fe6-be01-40db-94de-a835a32b052a admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:20:32.926837 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-5ca47fe6-be01-40db-94de-a835a32b052a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5f2113978b41471cbd3bfe4299e85c68', 'request_id': 'req-5ca47fe6-be01-40db-94de-a835a32b052a'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:20:32.932201 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-5ca47fe6-be01-40db-94de-a835a32b052a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5f2113978b41471cbd3bfe4299e85c68', 'request_id': 'req-5ca47fe6-be01-40db-94de-a835a32b052a'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:20:32.938269 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-5ca47fe6-be01-40db-94de-a835a32b052a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5f2113978b41471cbd3bfe4299e85c68', 'request_id': 'req-5ca47fe6-be01-40db-94de-a835a32b052a'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:20:33.011068 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-5ca47fe6-be01-40db-94de-a835a32b052a admin admin] The token cache is being invalidate because project 5f2113978b41471cbd3bfe4299e85c68 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81676) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 10 12:20:33.011364 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-5ca47fe6-be01-40db-94de-a835a32b052a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:20:33.012003 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-5ca47fe6-be01-40db-94de-a835a32b052a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:20:33.013509 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1241/2504] 10.222.0.46 () {58 vars in 1233 bytes} [Mon Oct 10 12:20:32 2022] DELETE /identity/v3/projects/5f2113978b41471cbd3bfe4299e85c68 => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:20:33.018560 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e6c254-e142-4db8-8a08-2bfca942e3e6 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:20:33.019110 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e6c254-e142-4db8-8a08-2bfca942e3e6 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:20:33.019229 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e6c254-e142-4db8-8a08-2bfca942e3e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:20:33.041378 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-39e6c254-e142-4db8-8a08-2bfca942e3e6 None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:20:33.047221 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-39e6c254-e142-4db8-8a08-2bfca942e3e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:20:33.080015 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:20:33.104208 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1264/2505] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:20:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:20:33.112502 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6f5f95-39d6-4888-a5f8-f70d70b1314d None None] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:20:33.112795 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6f5f95-39d6-4888-a5f8-f70d70b1314d None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:20:33.113073 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6f5f95-39d6-4888-a5f8-f70d70b1314d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:20:33.133116 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.auth.core [None req-df6f5f95-39d6-4888-a5f8-f70d70b1314d None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:20:33.134335 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-df6f5f95-39d6-4888-a5f8-f70d70b1314d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:20:33.143758 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1242/2506] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:20:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:20:33.148718 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e6c254-e142-4db8-8a08-2bfca942e3e6 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:20:33.159543 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e6c254-e142-4db8-8a08-2bfca942e3e6 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:20:33.165285 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe275555-f7f3-4c06-ac02-ab0d3b5dfd0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:20:33.165900 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe275555-f7f3-4c06-ac02-ab0d3b5dfd0b admin admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:20:33.166182 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe275555-f7f3-4c06-ac02-ab0d3b5dfd0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:20:33.166482 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe275555-f7f3-4c06-ac02-ab0d3b5dfd0b admin admin] PATH_INFO: `/v3/domains` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:20:33.167175 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe275555-f7f3-4c06-ac02-ab0d3b5dfd0b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 12:20:33.167475 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe275555-f7f3-4c06-ac02-ab0d3b5dfd0b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:20:33.169108 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe275555-f7f3-4c06-ac02-ab0d3b5dfd0b admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:20:33.181540 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1265/2507] 10.222.0.46 () {58 vars in 1153 bytes} [Mon Oct 10 12:20:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 12:20:33.186113 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6f5f95-39d6-4888-a5f8-f70d70b1314d None None] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:20:33.195371 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6f5f95-39d6-4888-a5f8-f70d70b1314d None None] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:20:33.200038 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac14b034-2523-405d-b9d4-ab56d881d148 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:20:33.200641 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac14b034-2523-405d-b9d4-ab56d881d148 admin admin] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:20:33.200918 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac14b034-2523-405d-b9d4-ab56d881d148 admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:20:33.201188 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac14b034-2523-405d-b9d4-ab56d881d148 admin admin] PATH_INFO: `/v3/projects` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:20:33.201863 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac14b034-2523-405d-b9d4-ab56d881d148 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:20:33.203810 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac14b034-2523-405d-b9d4-ab56d881d148 admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:20:33.304935 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1243/2508] 10.222.0.46 () {60 vars in 1153 bytes} [Mon Oct 10 12:20:33 2022] POST /identity/v3/projects => generated 377 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 10 12:20:33.309420 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe275555-f7f3-4c06-ac02-ab0d3b5dfd0b admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:20:33.320096 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe275555-f7f3-4c06-ac02-ab0d3b5dfd0b admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:20:33.378415 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c4418d-67da-44f0-9b37-b4fe15abc265 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:20:33.379135 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c4418d-67da-44f0-9b37-b4fe15abc265 admin admin] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:20:33.379921 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c4418d-67da-44f0-9b37-b4fe15abc265 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:20:33.380196 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c4418d-67da-44f0-9b37-b4fe15abc265 admin admin] PATH_INFO: `/v3/users` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:20:33.380838 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c4418d-67da-44f0-9b37-b4fe15abc265 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:20:33.382852 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c4418d-67da-44f0-9b37-b4fe15abc265 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:20:33.469633 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1266/2509] 10.222.0.46 () {60 vars in 1144 bytes} [Mon Oct 10 12:20:33 2022] POST /identity/v3/users => generated 289 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 12:20:33.482897 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac14b034-2523-405d-b9d4-ab56d881d148 admin admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:20:33.493697 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac14b034-2523-405d-b9d4-ab56d881d148 admin admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:20:33.498341 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4223b6-9c8c-47d6-a377-83076a4d3a26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:20:33.498974 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4223b6-9c8c-47d6-a377-83076a4d3a26 admin admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:20:33.499357 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4223b6-9c8c-47d6-a377-83076a4d3a26 admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:20:33.499638 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4223b6-9c8c-47d6-a377-83076a4d3a26 admin admin] PATH_INFO: `/v3/roles` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:20:33.500318 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4223b6-9c8c-47d6-a377-83076a4d3a26 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:20:33.501908 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4223b6-9c8c-47d6-a377-83076a4d3a26 admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:20:33.513011 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1244/2510] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 10 12:20:33 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 12:20:33.517336 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c4418d-67da-44f0-9b37-b4fe15abc265 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:20:33.528291 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c4418d-67da-44f0-9b37-b4fe15abc265 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:20:33.532903 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbd110f-256e-4556-896d-0b1abd14d8fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:20:33.533525 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbd110f-256e-4556-896d-0b1abd14d8fd admin admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:20:33.533800 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbd110f-256e-4556-896d-0b1abd14d8fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:20:33.534074 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbd110f-256e-4556-896d-0b1abd14d8fd admin admin] PATH_INFO: `/v3/roles` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:20:33.534897 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bbd110f-256e-4556-896d-0b1abd14d8fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:20:33.536371 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bbd110f-256e-4556-896d-0b1abd14d8fd admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:20:33.547154 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1267/2511] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 10 12:20:33 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 10 12:20:33.551438 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4223b6-9c8c-47d6-a377-83076a4d3a26 admin admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:20:33.561905 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4223b6-9c8c-47d6-a377-83076a4d3a26 admin admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:20:33.566567 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0a75c0-3cfd-46b3-a752-bb24c5b48c00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:20:33.567324 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0a75c0-3cfd-46b3-a752-bb24c5b48c00 admin admin] REQUEST_METHOD: `PUT` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:20:33.567606 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0a75c0-3cfd-46b3-a752-bb24c5b48c00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:20:33.568475 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0a75c0-3cfd-46b3-a752-bb24c5b48c00 admin admin] PATH_INFO: `/v3/projects/2b3f8734299a417c98fd38b85d9a1ed6/users/3f27f8a311744a04b9ef2a55232d9278/roles/b2cb2a6e818e47338ce071e6fedfc673` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:20:33.570884 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:20:33.584480 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0a75c0-3cfd-46b3-a752-bb24c5b48c00 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2b3f8734299a417c98fd38b85d9a1ed6, user_id=3f27f8a311744a04b9ef2a55232d9278, role_id=b2cb2a6e818e47338ce071e6fedfc673)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:20:33.587267 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0a75c0-3cfd-46b3-a752-bb24c5b48c00 admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:20:33.652499 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1245/2512] 10.222.0.46 () {60 vars in 1483 bytes} [Mon Oct 10 12:20:33 2022] PUT /identity/v3/projects/2b3f8734299a417c98fd38b85d9a1ed6/users/3f27f8a311744a04b9ef2a55232d9278/roles/b2cb2a6e818e47338ce071e6fedfc673 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:20:33.663343 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92de59c9-0702-4ba4-a07d-72b59a04c27d None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:20:33.663645 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92de59c9-0702-4ba4-a07d-72b59a04c27d None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:20:33.663898 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92de59c9-0702-4ba4-a07d-72b59a04c27d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:20:33.694935 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-92de59c9-0702-4ba4-a07d-72b59a04c27d None None] MFA Rules not processed for user `3f27f8a311744a04b9ef2a55232d9278`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:20:33.696293 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-92de59c9-0702-4ba4-a07d-72b59a04c27d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:20:33.764185 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1268/2513] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:20:33 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:20:33.768721 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0a75c0-3cfd-46b3-a752-bb24c5b48c00 admin admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:20:33.831253 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0a75c0-3cfd-46b3-a752-bb24c5b48c00 admin admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:20:33.836058 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e3e68c-52a8-415d-893a-11b4d7afaa03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:20:33.836691 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e3e68c-52a8-415d-893a-11b4d7afaa03 admin admin] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:20:33.836972 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e3e68c-52a8-415d-893a-11b4d7afaa03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:20:33.837247 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e3e68c-52a8-415d-893a-11b4d7afaa03 admin admin] PATH_INFO: `/v3/projects` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:20:33.838148 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e3e68c-52a8-415d-893a-11b4d7afaa03 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:20:33.840120 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e3e68c-52a8-415d-893a-11b4d7afaa03 admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:20:33.924699 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1246/2514] 10.222.0.46 () {60 vars in 1153 bytes} [Mon Oct 10 12:20:33 2022] POST /identity/v3/projects => generated 381 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 12:20:33.929224 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92de59c9-0702-4ba4-a07d-72b59a04c27d None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:20:33.939923 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92de59c9-0702-4ba4-a07d-72b59a04c27d None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:20:33.976867 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:20:34.001089 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cceb01e-286d-4e07-82c9-79bc1b18a324 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:20:34.001728 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cceb01e-286d-4e07-82c9-79bc1b18a324 admin admin] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:20:34.002010 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cceb01e-286d-4e07-82c9-79bc1b18a324 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:20:34.002283 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cceb01e-286d-4e07-82c9-79bc1b18a324 admin admin] PATH_INFO: `/v3/users` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:20:34.002960 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cceb01e-286d-4e07-82c9-79bc1b18a324 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:20:34.004976 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cceb01e-286d-4e07-82c9-79bc1b18a324 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:20:34.126481 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1269/2515] 10.222.0.46 () {60 vars in 1144 bytes} [Mon Oct 10 12:20:33 2022] POST /identity/v3/users => generated 297 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 12:20:34.130659 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e3e68c-52a8-415d-893a-11b4d7afaa03 admin admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:20:34.141077 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e3e68c-52a8-415d-893a-11b4d7afaa03 admin admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:20:34.145838 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5507819-8eaf-4216-a031-9509de06bcee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:20:34.146438 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5507819-8eaf-4216-a031-9509de06bcee admin admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:20:34.146749 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5507819-8eaf-4216-a031-9509de06bcee admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:20:34.147078 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5507819-8eaf-4216-a031-9509de06bcee admin admin] PATH_INFO: `/v3/roles` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:20:34.147727 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5507819-8eaf-4216-a031-9509de06bcee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:20:34.149342 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5507819-8eaf-4216-a031-9509de06bcee admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:20:34.160224 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1247/2516] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 10 12:20:34 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 12:20:34.165242 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cceb01e-286d-4e07-82c9-79bc1b18a324 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:20:34.175703 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cceb01e-286d-4e07-82c9-79bc1b18a324 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:20:34.181031 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b89789d-b2db-40ab-bc7b-4aeea7bd39c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:20:34.181692 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b89789d-b2db-40ab-bc7b-4aeea7bd39c0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:20:34.181964 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b89789d-b2db-40ab-bc7b-4aeea7bd39c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:20:34.182522 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b89789d-b2db-40ab-bc7b-4aeea7bd39c0 admin admin] PATH_INFO: `/v3/projects/001921ff584f403099fb33aac7927550/users/d4ea50873d644ff391ce40808e7a94f0/roles/0d822a0674b6495fac623877a3b38025` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:20:34.197429 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b89789d-b2db-40ab-bc7b-4aeea7bd39c0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=001921ff584f403099fb33aac7927550, user_id=d4ea50873d644ff391ce40808e7a94f0, role_id=0d822a0674b6495fac623877a3b38025)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:20:34.200109 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b89789d-b2db-40ab-bc7b-4aeea7bd39c0 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:20:34.264458 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1270/2517] 10.222.0.46 () {60 vars in 1483 bytes} [Mon Oct 10 12:20:34 2022] PUT /identity/v3/projects/001921ff584f403099fb33aac7927550/users/d4ea50873d644ff391ce40808e7a94f0/roles/0d822a0674b6495fac623877a3b38025 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:20:34.268706 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5507819-8eaf-4216-a031-9509de06bcee admin admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:20:34.279292 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5507819-8eaf-4216-a031-9509de06bcee admin admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:20:34.335469 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b85970-c033-4d07-b4f7-cee16f40ae46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:20:34.336075 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b85970-c033-4d07-b4f7-cee16f40ae46 admin admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:20:34.336356 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b85970-c033-4d07-b4f7-cee16f40ae46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:20:34.336630 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b85970-c033-4d07-b4f7-cee16f40ae46 admin admin] PATH_INFO: `/v3/roles` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:20:34.337275 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b85970-c033-4d07-b4f7-cee16f40ae46 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:20:34.339174 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b85970-c033-4d07-b4f7-cee16f40ae46 admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:20:34.350188 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1248/2518] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 10 12:20:34 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 12:20:34.354730 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b89789d-b2db-40ab-bc7b-4aeea7bd39c0 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:20:34.365994 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b89789d-b2db-40ab-bc7b-4aeea7bd39c0 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:20:34.370858 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ab0054-0b21-4a2d-a644-20773ac70f3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:20:34.371530 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ab0054-0b21-4a2d-a644-20773ac70f3d admin admin] REQUEST_METHOD: `PUT` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:20:34.371812 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ab0054-0b21-4a2d-a644-20773ac70f3d admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:20:34.372123 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ab0054-0b21-4a2d-a644-20773ac70f3d admin admin] PATH_INFO: `/v3/projects/001921ff584f403099fb33aac7927550/users/d4ea50873d644ff391ce40808e7a94f0/roles/0d822a0674b6495fac623877a3b38025` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:20:34.374051 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ab0054-0b21-4a2d-a644-20773ac70f3d admin admin] RBAC: Authorizing `identity:create_grant(project_id=001921ff584f403099fb33aac7927550, user_id=d4ea50873d644ff391ce40808e7a94f0, role_id=0d822a0674b6495fac623877a3b38025)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:20:34.376827 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ab0054-0b21-4a2d-a644-20773ac70f3d admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:20:34.387710 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1271/2519] 10.222.0.46 () {60 vars in 1483 bytes} [Mon Oct 10 12:20:34 2022] PUT /identity/v3/projects/001921ff584f403099fb33aac7927550/users/d4ea50873d644ff391ce40808e7a94f0/roles/0d822a0674b6495fac623877a3b38025 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:20:34.400115 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d728325-b59f-4297-9c40-6544fb34a43d None None] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:20:34.400434 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d728325-b59f-4297-9c40-6544fb34a43d None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:20:34.400670 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d728325-b59f-4297-9c40-6544fb34a43d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:20:34.430066 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.auth.core [None req-8d728325-b59f-4297-9c40-6544fb34a43d None None] MFA Rules not processed for user `d4ea50873d644ff391ce40808e7a94f0`. Rule list: `[]` (Enabled: `True`). {{(pid=81676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:20:34.431236 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-8d728325-b59f-4297-9c40-6544fb34a43d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:20:34.476053 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:20:34.504306 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1249/2520] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:20:34 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:20:34.516293 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ab0054-0b21-4a2d-a644-20773ac70f3d admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:20:34.518024 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-b5ab0054-0b21-4a2d-a644-20773ac70f3d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:20:34.577064 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ab0054-0b21-4a2d-a644-20773ac70f3d admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:20:34.580781 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4bb7fd-0811-4fde-bfab-17a47256f1bd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3ae093470484b93b23dcfae1657df9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad3772f7b6a47c28b7ba37a9316a3c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:20:34.581414 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4bb7fd-0811-4fde-bfab-17a47256f1bd service cinder] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:20:34.581699 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4bb7fd-0811-4fde-bfab-17a47256f1bd service cinder] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:20:34.581999 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4bb7fd-0811-4fde-bfab-17a47256f1bd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:20:34.592118 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4bb7fd-0811-4fde-bfab-17a47256f1bd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:20:34.593532 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4bb7fd-0811-4fde-bfab-17a47256f1bd service cinder] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:20:34.603653 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1272/2521] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 10 12:20:34 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 10 12:20:35.102269 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d728325-b59f-4297-9c40-6544fb34a43d None None] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:20:35.169264 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d728325-b59f-4297-9c40-6544fb34a43d None None] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:20:35.173822 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1571bb43-dc93-4ca3-b7ca-c3152fc7eccf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:20:35.174452 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1571bb43-dc93-4ca3-b7ca-c3152fc7eccf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:20:35.174765 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1571bb43-dc93-4ca3-b7ca-c3152fc7eccf admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:20:35.175040 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1571bb43-dc93-4ca3-b7ca-c3152fc7eccf admin admin] PATH_INFO: `/v3/users/3f27f8a311744a04b9ef2a55232d9278` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:20:35.176184 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1571bb43-dc93-4ca3-b7ca-c3152fc7eccf admin admin] RBAC: Authorizing `identity:delete_user(user_id=3f27f8a311744a04b9ef2a55232d9278)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:20:35.178575 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1571bb43-dc93-4ca3-b7ca-c3152fc7eccf admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:20:35.366469 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-1571bb43-dc93-4ca3-b7ca-c3152fc7eccf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3f27f8a311744a04b9ef2a55232d9278'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:20:35.373792 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-1571bb43-dc93-4ca3-b7ca-c3152fc7eccf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3f27f8a311744a04b9ef2a55232d9278'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:20:35.374459 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-1571bb43-dc93-4ca3-b7ca-c3152fc7eccf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3f27f8a311744a04b9ef2a55232d9278'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:20:35.434202 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-1571bb43-dc93-4ca3-b7ca-c3152fc7eccf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3f27f8a311744a04b9ef2a55232d9278'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:20:35.447933 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-1571bb43-dc93-4ca3-b7ca-c3152fc7eccf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3f27f8a311744a04b9ef2a55232d9278'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:20:35.454078 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1250/2522] 10.222.0.46 () {58 vars in 1224 bytes} [Mon Oct 10 12:20:35 2022] DELETE /identity/v3/users/3f27f8a311744a04b9ef2a55232d9278 => generated 0 bytes in 352 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:20:35.458687 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7a7139-64bc-4a99-bb78-a70a6d4b9134 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:20:35.459165 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7a7139-64bc-4a99-bb78-a70a6d4b9134 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:20:35.459332 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7a7139-64bc-4a99-bb78-a70a6d4b9134 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:20:35.478658 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-ac7a7139-64bc-4a99-bb78-a70a6d4b9134 None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:20:35.479906 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-ac7a7139-64bc-4a99-bb78-a70a6d4b9134 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:20:35.541807 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1273/2523] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:20:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:20:35.553875 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1571bb43-dc93-4ca3-b7ca-c3152fc7eccf admin admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:20:35.556486 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-1571bb43-dc93-4ca3-b7ca-c3152fc7eccf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:20:35.613668 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1571bb43-dc93-4ca3-b7ca-c3152fc7eccf admin admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:20:35.618113 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e953d02e-40e3-4d92-84a2-3965a293fb50 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58b1c7c85302421cba823ace4201861f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad3772f7b6a47c28b7ba37a9316a3c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:20:35.618677 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e953d02e-40e3-4d92-84a2-3965a293fb50 service neutron] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:20:35.618963 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e953d02e-40e3-4d92-84a2-3965a293fb50 service neutron] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:20:35.619238 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e953d02e-40e3-4d92-84a2-3965a293fb50 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:20:35.628843 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e953d02e-40e3-4d92-84a2-3965a293fb50 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:20:35.630186 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e953d02e-40e3-4d92-84a2-3965a293fb50 service neutron] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:20:35.635407 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:20:35.640555 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1251/2524] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 10 12:20:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:20:36.062173 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7a7139-64bc-4a99-bb78-a70a6d4b9134 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:20:36.063620 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-ac7a7139-64bc-4a99-bb78-a70a6d4b9134 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:20:36.077965 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7a7139-64bc-4a99-bb78-a70a6d4b9134 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:20:36.082679 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff30139-236f-4847-a731-98b342f3a92d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:20:36.083327 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff30139-236f-4847-a731-98b342f3a92d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:20:36.084106 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff30139-236f-4847-a731-98b342f3a92d admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:20:36.084372 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff30139-236f-4847-a731-98b342f3a92d admin admin] PATH_INFO: `/v3/projects/2b3f8734299a417c98fd38b85d9a1ed6` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:20:36.086575 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff30139-236f-4847-a731-98b342f3a92d admin admin] RBAC: Authorizing `identity:delete_project(project_id=2b3f8734299a417c98fd38b85d9a1ed6)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:20:36.089104 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff30139-236f-4847-a731-98b342f3a92d admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:20:36.171705 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=93,keystone:INSERT=5,keystone:DELETE=2 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:20:36.187040 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-4ff30139-236f-4847-a731-98b342f3a92d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2b3f8734299a417c98fd38b85d9a1ed6', 'request_id': 'req-4ff30139-236f-4847-a731-98b342f3a92d'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:20:36.192704 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-4ff30139-236f-4847-a731-98b342f3a92d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2b3f8734299a417c98fd38b85d9a1ed6', 'request_id': 'req-4ff30139-236f-4847-a731-98b342f3a92d'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:20:36.199185 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-4ff30139-236f-4847-a731-98b342f3a92d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2b3f8734299a417c98fd38b85d9a1ed6', 'request_id': 'req-4ff30139-236f-4847-a731-98b342f3a92d'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:20:36.257432 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-4ff30139-236f-4847-a731-98b342f3a92d admin admin] The token cache is being invalidate because project 2b3f8734299a417c98fd38b85d9a1ed6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81677) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 10 12:20:36.257727 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-4ff30139-236f-4847-a731-98b342f3a92d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:20:36.258419 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-4ff30139-236f-4847-a731-98b342f3a92d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:20:36.259960 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1274/2525] 10.222.0.46 () {58 vars in 1233 bytes} [Mon Oct 10 12:20:36 2022] DELETE /identity/v3/projects/2b3f8734299a417c98fd38b85d9a1ed6 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:20:36.264215 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e953d02e-40e3-4d92-84a2-3965a293fb50 service neutron] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:20:36.274973 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e953d02e-40e3-4d92-84a2-3965a293fb50 service neutron] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:20:36.335571 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f811713e-9481-45cf-ab16-84096be53d85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:20:36.336240 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f811713e-9481-45cf-ab16-84096be53d85 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:20:36.336513 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f811713e-9481-45cf-ab16-84096be53d85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:20:36.336781 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f811713e-9481-45cf-ab16-84096be53d85 admin admin] PATH_INFO: `/v3/users/d4ea50873d644ff391ce40808e7a94f0` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:20:36.338215 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f811713e-9481-45cf-ab16-84096be53d85 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d4ea50873d644ff391ce40808e7a94f0)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:20:36.340249 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f811713e-9481-45cf-ab16-84096be53d85 admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:20:36.549406 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-f811713e-9481-45cf-ab16-84096be53d85 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd4ea50873d644ff391ce40808e7a94f0'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:20:36.557047 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-f811713e-9481-45cf-ab16-84096be53d85 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd4ea50873d644ff391ce40808e7a94f0'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:20:36.557740 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-f811713e-9481-45cf-ab16-84096be53d85 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd4ea50873d644ff391ce40808e7a94f0'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:20:36.651626 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-f811713e-9481-45cf-ab16-84096be53d85 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd4ea50873d644ff391ce40808e7a94f0'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:20:36.665363 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-f811713e-9481-45cf-ab16-84096be53d85 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd4ea50873d644ff391ce40808e7a94f0'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:20:36.678897 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1252/2526] 10.222.0.46 () {58 vars in 1224 bytes} [Mon Oct 10 12:20:36 2022] DELETE /identity/v3/users/d4ea50873d644ff391ce40808e7a94f0 => generated 0 bytes in 415 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:20:37.162355 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff30139-236f-4847-a731-98b342f3a92d admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:20:37.163880 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-4ff30139-236f-4847-a731-98b342f3a92d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:20:37.179234 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff30139-236f-4847-a731-98b342f3a92d admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:20:37.238833 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e74ef01-0bf9-4bc4-ab7e-dd29dfd636b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:20:37.239426 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e74ef01-0bf9-4bc4-ab7e-dd29dfd636b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:20:37.240314 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e74ef01-0bf9-4bc4-ab7e-dd29dfd636b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:20:37.240573 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e74ef01-0bf9-4bc4-ab7e-dd29dfd636b4 admin admin] PATH_INFO: `/v3/projects/001921ff584f403099fb33aac7927550` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:20:37.241718 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e74ef01-0bf9-4bc4-ab7e-dd29dfd636b4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=001921ff584f403099fb33aac7927550)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:20:37.243708 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e74ef01-0bf9-4bc4-ab7e-dd29dfd636b4 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:20:37.347742 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-1e74ef01-0bf9-4bc4-ab7e-dd29dfd636b4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '001921ff584f403099fb33aac7927550', 'request_id': 'req-1e74ef01-0bf9-4bc4-ab7e-dd29dfd636b4'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:20:37.353860 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-1e74ef01-0bf9-4bc4-ab7e-dd29dfd636b4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '001921ff584f403099fb33aac7927550', 'request_id': 'req-1e74ef01-0bf9-4bc4-ab7e-dd29dfd636b4'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:20:37.359940 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-1e74ef01-0bf9-4bc4-ab7e-dd29dfd636b4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '001921ff584f403099fb33aac7927550', 'request_id': 'req-1e74ef01-0bf9-4bc4-ab7e-dd29dfd636b4'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:20:37.419842 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-1e74ef01-0bf9-4bc4-ab7e-dd29dfd636b4 admin admin] The token cache is being invalidate because project 001921ff584f403099fb33aac7927550 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81677) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 10 12:20:37.420040 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-1e74ef01-0bf9-4bc4-ab7e-dd29dfd636b4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:20:37.420657 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-1e74ef01-0bf9-4bc4-ab7e-dd29dfd636b4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:20:37.422190 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1275/2527] 10.222.0.46 () {58 vars in 1233 bytes} [Mon Oct 10 12:20:37 2022] DELETE /identity/v3/projects/001921ff584f403099fb33aac7927550 => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:20:37.427206 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8058773-5942-4041-a696-d36c31449d3c None None] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:20:37.428401 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8058773-5942-4041-a696-d36c31449d3c None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:20:37.428401 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8058773-5942-4041-a696-d36c31449d3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:20:37.447460 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.auth.core [None req-f8058773-5942-4041-a696-d36c31449d3c None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:20:37.448685 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-f8058773-5942-4041-a696-d36c31449d3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:20:37.509663 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1253/2528] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:20:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:20:37.517716 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ec0bc1-2554-4fca-b877-922aaba8d05c None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:20:37.518004 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ec0bc1-2554-4fca-b877-922aaba8d05c None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:20:37.518317 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ec0bc1-2554-4fca-b877-922aaba8d05c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:20:37.539410 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-39ec0bc1-2554-4fca-b877-922aaba8d05c None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:20:37.540614 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-39ec0bc1-2554-4fca-b877-922aaba8d05c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:20:37.548490 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1276/2529] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:20:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:20:37.552832 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8058773-5942-4041-a696-d36c31449d3c None None] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:20:37.562600 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8058773-5942-4041-a696-d36c31449d3c None None] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:20:37.567671 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ecda411-cdb3-4214-bba4-0036ffe8e8a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:20:37.568278 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecda411-cdb3-4214-bba4-0036ffe8e8a0 admin admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:20:37.568559 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecda411-cdb3-4214-bba4-0036ffe8e8a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:20:37.568832 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecda411-cdb3-4214-bba4-0036ffe8e8a0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:20:37.569496 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ecda411-cdb3-4214-bba4-0036ffe8e8a0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 12:20:37.569796 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ecda411-cdb3-4214-bba4-0036ffe8e8a0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:20:37.571451 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ecda411-cdb3-4214-bba4-0036ffe8e8a0 admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:20:37.584022 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1254/2530] 10.222.0.46 () {58 vars in 1153 bytes} [Mon Oct 10 12:20:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 12:20:37.588772 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ec0bc1-2554-4fca-b877-922aaba8d05c None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:20:37.598126 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ec0bc1-2554-4fca-b877-922aaba8d05c None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:20:37.603380 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e108d39-1b0f-4548-8721-2b04ae47f635 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:20:37.604001 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e108d39-1b0f-4548-8721-2b04ae47f635 admin admin] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:20:37.604281 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e108d39-1b0f-4548-8721-2b04ae47f635 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:20:37.604551 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e108d39-1b0f-4548-8721-2b04ae47f635 admin admin] PATH_INFO: `/v3/projects` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:20:37.605200 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e108d39-1b0f-4548-8721-2b04ae47f635 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:20:37.607142 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e108d39-1b0f-4548-8721-2b04ae47f635 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:20:37.711307 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1277/2531] 10.222.0.46 () {60 vars in 1153 bytes} [Mon Oct 10 12:20:37 2022] POST /identity/v3/projects => generated 385 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 12:20:37.715635 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ecda411-cdb3-4214-bba4-0036ffe8e8a0 admin admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:20:37.726308 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ecda411-cdb3-4214-bba4-0036ffe8e8a0 admin admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:20:37.729946 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:20:37.783689 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c7ed4f-d059-4a2e-af0f-f1986af1b7d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:20:37.784301 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c7ed4f-d059-4a2e-af0f-f1986af1b7d9 admin admin] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:20:37.784581 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c7ed4f-d059-4a2e-af0f-f1986af1b7d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:20:37.784852 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c7ed4f-d059-4a2e-af0f-f1986af1b7d9 admin admin] PATH_INFO: `/v3/users` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:20:37.785488 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c7ed4f-d059-4a2e-af0f-f1986af1b7d9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:20:37.787441 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c7ed4f-d059-4a2e-af0f-f1986af1b7d9 admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:20:37.895429 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1255/2532] 10.222.0.46 () {60 vars in 1144 bytes} [Mon Oct 10 12:20:37 2022] POST /identity/v3/users => generated 293 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 12:20:37.900415 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e108d39-1b0f-4548-8721-2b04ae47f635 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:20:37.910902 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e108d39-1b0f-4548-8721-2b04ae47f635 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:20:37.915587 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e8cbb8-e0bd-42ae-a700-d8d37a61fa4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:20:37.916192 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e8cbb8-e0bd-42ae-a700-d8d37a61fa4d admin admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:20:37.916469 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e8cbb8-e0bd-42ae-a700-d8d37a61fa4d admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:20:37.916739 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e8cbb8-e0bd-42ae-a700-d8d37a61fa4d admin admin] PATH_INFO: `/v3/roles` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:20:37.917379 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e8cbb8-e0bd-42ae-a700-d8d37a61fa4d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:20:37.919192 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e8cbb8-e0bd-42ae-a700-d8d37a61fa4d admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:20:37.929602 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1278/2533] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 10 12:20:37 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 12:20:37.934018 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c7ed4f-d059-4a2e-af0f-f1986af1b7d9 admin admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:20:37.945782 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c7ed4f-d059-4a2e-af0f-f1986af1b7d9 admin admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:20:37.951002 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff4c52a-c0e4-4ff1-9372-4577ba8c3e3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:20:37.951700 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff4c52a-c0e4-4ff1-9372-4577ba8c3e3e admin admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:20:37.952156 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff4c52a-c0e4-4ff1-9372-4577ba8c3e3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:20:37.952438 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff4c52a-c0e4-4ff1-9372-4577ba8c3e3e admin admin] PATH_INFO: `/v3/roles` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:20:37.953101 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff4c52a-c0e4-4ff1-9372-4577ba8c3e3e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:20:37.954761 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff4c52a-c0e4-4ff1-9372-4577ba8c3e3e admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:20:37.966736 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1256/2534] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 10 12:20:37 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 12:20:37.970922 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e8cbb8-e0bd-42ae-a700-d8d37a61fa4d admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:20:37.981287 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e8cbb8-e0bd-42ae-a700-d8d37a61fa4d admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:20:37.985997 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918fa1a1-4536-4fb7-a240-107953cbe250 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:20:37.986650 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918fa1a1-4536-4fb7-a240-107953cbe250 admin admin] REQUEST_METHOD: `PUT` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:20:37.986951 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918fa1a1-4536-4fb7-a240-107953cbe250 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:20:37.987226 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918fa1a1-4536-4fb7-a240-107953cbe250 admin admin] PATH_INFO: `/v3/projects/93fe89d358aa44afb40f67f46596c571/users/5e63cfd796074ca69f78e18aec2a4c85/roles/b2cb2a6e818e47338ce071e6fedfc673` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:20:37.989527 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:20:38.002179 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918fa1a1-4536-4fb7-a240-107953cbe250 admin admin] RBAC: Authorizing `identity:create_grant(project_id=93fe89d358aa44afb40f67f46596c571, user_id=5e63cfd796074ca69f78e18aec2a4c85, role_id=b2cb2a6e818e47338ce071e6fedfc673)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:20:38.004838 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918fa1a1-4536-4fb7-a240-107953cbe250 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:20:38.090630 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1279/2535] 10.222.0.46 () {60 vars in 1483 bytes} [Mon Oct 10 12:20:37 2022] PUT /identity/v3/projects/93fe89d358aa44afb40f67f46596c571/users/5e63cfd796074ca69f78e18aec2a4c85/roles/b2cb2a6e818e47338ce071e6fedfc673 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:20:38.102822 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def30746-043f-4251-8009-86cf8f313413 None None] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:20:38.103222 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def30746-043f-4251-8009-86cf8f313413 None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:20:38.103508 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def30746-043f-4251-8009-86cf8f313413 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:20:38.132841 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.auth.core [None req-def30746-043f-4251-8009-86cf8f313413 None None] MFA Rules not processed for user `5e63cfd796074ca69f78e18aec2a4c85`. Rule list: `[]` (Enabled: `True`). {{(pid=81676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:20:38.134062 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-def30746-043f-4251-8009-86cf8f313413 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:20:38.198421 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1257/2536] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:20:38 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:20:38.203413 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918fa1a1-4536-4fb7-a240-107953cbe250 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:20:38.270969 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918fa1a1-4536-4fb7-a240-107953cbe250 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:20:38.272151 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed31eef-9ca4-4553-9fbc-62f068262e9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:20:38.272771 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed31eef-9ca4-4553-9fbc-62f068262e9c admin admin] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:20:38.273121 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed31eef-9ca4-4553-9fbc-62f068262e9c admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:20:38.273319 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed31eef-9ca4-4553-9fbc-62f068262e9c admin admin] PATH_INFO: `/v3/projects` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:20:38.273955 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed31eef-9ca4-4553-9fbc-62f068262e9c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:20:38.275880 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed31eef-9ca4-4553-9fbc-62f068262e9c admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:20:38.379562 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1280/2537] 10.222.0.46 () {60 vars in 1153 bytes} [Mon Oct 10 12:20:38 2022] POST /identity/v3/projects => generated 383 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 12:20:38.384629 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def30746-043f-4251-8009-86cf8f313413 None None] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:20:38.396026 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def30746-043f-4251-8009-86cf8f313413 None None] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:20:38.459811 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8c53d7-e3db-4b92-a527-2be58cfac19e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:20:38.460470 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8c53d7-e3db-4b92-a527-2be58cfac19e admin admin] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:20:38.460695 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8c53d7-e3db-4b92-a527-2be58cfac19e admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:20:38.460974 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8c53d7-e3db-4b92-a527-2be58cfac19e admin admin] PATH_INFO: `/v3/users` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:20:38.461637 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8c53d7-e3db-4b92-a527-2be58cfac19e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:20:38.463897 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8c53d7-e3db-4b92-a527-2be58cfac19e admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:20:38.570352 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1258/2538] 10.222.0.46 () {60 vars in 1144 bytes} [Mon Oct 10 12:20:38 2022] POST /identity/v3/users => generated 292 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 12:20:38.574283 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed31eef-9ca4-4553-9fbc-62f068262e9c admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:20:38.585358 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed31eef-9ca4-4553-9fbc-62f068262e9c admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:20:38.590272 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79494dd7-48df-4d10-abf5-2c6d0ab93619 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:20:38.590925 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79494dd7-48df-4d10-abf5-2c6d0ab93619 admin admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:20:38.591209 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79494dd7-48df-4d10-abf5-2c6d0ab93619 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:20:38.591488 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79494dd7-48df-4d10-abf5-2c6d0ab93619 admin admin] PATH_INFO: `/v3/roles` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:20:38.592139 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79494dd7-48df-4d10-abf5-2c6d0ab93619 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:20:38.593843 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79494dd7-48df-4d10-abf5-2c6d0ab93619 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:20:38.606138 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1281/2539] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 10 12:20:38 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 12:20:38.610620 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8c53d7-e3db-4b92-a527-2be58cfac19e admin admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:20:38.617745 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:20:38.621857 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8c53d7-e3db-4b92-a527-2be58cfac19e admin admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:20:38.626551 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbde279-b645-4532-95c7-886da76bf194 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:20:38.627178 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbde279-b645-4532-95c7-886da76bf194 admin admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:20:38.627455 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbde279-b645-4532-95c7-886da76bf194 admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:20:38.627741 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbde279-b645-4532-95c7-886da76bf194 admin admin] PATH_INFO: `/v3/roles` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:20:38.628362 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abbde279-b645-4532-95c7-886da76bf194 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:20:38.629979 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abbde279-b645-4532-95c7-886da76bf194 admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:20:38.641527 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1259/2540] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 10 12:20:38 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 10 12:20:38.645774 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79494dd7-48df-4d10-abf5-2c6d0ab93619 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:20:38.656366 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79494dd7-48df-4d10-abf5-2c6d0ab93619 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:20:38.660949 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3d6f89-d2fc-4740-8c9a-c6c06b9a2719 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:20:38.662130 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3d6f89-d2fc-4740-8c9a-c6c06b9a2719 admin admin] REQUEST_METHOD: `PUT` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:20:38.662410 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3d6f89-d2fc-4740-8c9a-c6c06b9a2719 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:20:38.662721 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3d6f89-d2fc-4740-8c9a-c6c06b9a2719 admin admin] PATH_INFO: `/v3/projects/db1fef7ea1c74d47b44e6ba7ce27f7de/users/8dd1d4c3b9844dc5b3c1d7e3407547b2/roles/b2cb2a6e818e47338ce071e6fedfc673` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:20:38.677889 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3d6f89-d2fc-4740-8c9a-c6c06b9a2719 admin admin] RBAC: Authorizing `identity:create_grant(project_id=db1fef7ea1c74d47b44e6ba7ce27f7de, user_id=8dd1d4c3b9844dc5b3c1d7e3407547b2, role_id=b2cb2a6e818e47338ce071e6fedfc673)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:20:38.680552 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3d6f89-d2fc-4740-8c9a-c6c06b9a2719 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:20:38.754946 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1282/2541] 10.222.0.46 () {60 vars in 1483 bytes} [Mon Oct 10 12:20:38 2022] PUT /identity/v3/projects/db1fef7ea1c74d47b44e6ba7ce27f7de/users/8dd1d4c3b9844dc5b3c1d7e3407547b2/roles/b2cb2a6e818e47338ce071e6fedfc673 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:20:38.765585 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71674a05-b8a3-418a-9f64-a0e6e77a2246 None None] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:20:38.765873 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71674a05-b8a3-418a-9f64-a0e6e77a2246 None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:20:38.766146 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71674a05-b8a3-418a-9f64-a0e6e77a2246 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:20:38.795525 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.auth.core [None req-71674a05-b8a3-418a-9f64-a0e6e77a2246 None None] MFA Rules not processed for user `8dd1d4c3b9844dc5b3c1d7e3407547b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:20:38.796738 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-71674a05-b8a3-418a-9f64-a0e6e77a2246 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:20:38.862781 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1260/2542] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:20:38 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:20:38.868193 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3d6f89-d2fc-4740-8c9a-c6c06b9a2719 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:20:38.934353 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3d6f89-d2fc-4740-8c9a-c6c06b9a2719 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:20:38.939198 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c96e07-22aa-4e7d-9b19-86990bb434f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:20:38.939811 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c96e07-22aa-4e7d-9b19-86990bb434f6 admin admin] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:20:38.940094 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c96e07-22aa-4e7d-9b19-86990bb434f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:20:38.940373 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c96e07-22aa-4e7d-9b19-86990bb434f6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:20:38.941020 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c96e07-22aa-4e7d-9b19-86990bb434f6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:20:38.942992 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c96e07-22aa-4e7d-9b19-86990bb434f6 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:20:39.032937 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1283/2543] 10.222.0.46 () {60 vars in 1153 bytes} [Mon Oct 10 12:20:38 2022] POST /identity/v3/projects => generated 383 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 12:20:39.037945 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71674a05-b8a3-418a-9f64-a0e6e77a2246 None None] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:20:39.049115 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71674a05-b8a3-418a-9f64-a0e6e77a2246 None None] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:20:39.108226 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a5dc7e-12e2-4ef5-839c-673fbcd5a3bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:20:39.108747 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a5dc7e-12e2-4ef5-839c-673fbcd5a3bd admin admin] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:20:39.109032 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a5dc7e-12e2-4ef5-839c-673fbcd5a3bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:20:39.109309 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a5dc7e-12e2-4ef5-839c-673fbcd5a3bd admin admin] PATH_INFO: `/v3/users` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:20:39.109951 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a5dc7e-12e2-4ef5-839c-673fbcd5a3bd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:20:39.111918 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a5dc7e-12e2-4ef5-839c-673fbcd5a3bd admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:20:39.223531 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1261/2544] 10.222.0.46 () {60 vars in 1144 bytes} [Mon Oct 10 12:20:39 2022] POST /identity/v3/users => generated 298 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 12:20:39.228236 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c96e07-22aa-4e7d-9b19-86990bb434f6 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:20:39.240240 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c96e07-22aa-4e7d-9b19-86990bb434f6 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:20:39.245056 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5770d9e7-3654-4b8e-b455-ff44cef6833f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:20:39.245677 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5770d9e7-3654-4b8e-b455-ff44cef6833f admin admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:20:39.245961 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5770d9e7-3654-4b8e-b455-ff44cef6833f admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:20:39.246270 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5770d9e7-3654-4b8e-b455-ff44cef6833f admin admin] PATH_INFO: `/v3/roles` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:20:39.246980 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5770d9e7-3654-4b8e-b455-ff44cef6833f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:20:39.248585 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5770d9e7-3654-4b8e-b455-ff44cef6833f admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:20:39.260043 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1284/2545] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 10 12:20:39 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 12:20:39.264329 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a5dc7e-12e2-4ef5-839c-673fbcd5a3bd admin admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:20:39.275340 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a5dc7e-12e2-4ef5-839c-673fbcd5a3bd admin admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:20:39.280089 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5df070-0eec-4b56-a0cd-7dfe30b52768 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:20:39.280746 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5df070-0eec-4b56-a0cd-7dfe30b52768 admin admin] REQUEST_METHOD: `PUT` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:20:39.281031 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5df070-0eec-4b56-a0cd-7dfe30b52768 admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:20:39.281310 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5df070-0eec-4b56-a0cd-7dfe30b52768 admin admin] PATH_INFO: `/v3/projects/cb5dbfe7641b4b42b4aed0ca4dcaa808/users/219e27f423b64f2d95c4e3b163ea38ed/roles/0d822a0674b6495fac623877a3b38025` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:20:39.296445 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5df070-0eec-4b56-a0cd-7dfe30b52768 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cb5dbfe7641b4b42b4aed0ca4dcaa808, user_id=219e27f423b64f2d95c4e3b163ea38ed, role_id=0d822a0674b6495fac623877a3b38025)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:20:39.299154 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5df070-0eec-4b56-a0cd-7dfe30b52768 admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:20:39.365712 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1262/2546] 10.222.0.46 () {60 vars in 1483 bytes} [Mon Oct 10 12:20:39 2022] PUT /identity/v3/projects/cb5dbfe7641b4b42b4aed0ca4dcaa808/users/219e27f423b64f2d95c4e3b163ea38ed/roles/0d822a0674b6495fac623877a3b38025 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:20:39.369640 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5770d9e7-3654-4b8e-b455-ff44cef6833f admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:20:39.379686 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5770d9e7-3654-4b8e-b455-ff44cef6833f admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:20:39.383774 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:20:39.436001 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd69c89-5b81-4218-ac1c-c15c08bdb463 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:20:39.436607 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd69c89-5b81-4218-ac1c-c15c08bdb463 admin admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:20:39.436897 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd69c89-5b81-4218-ac1c-c15c08bdb463 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:20:39.437171 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd69c89-5b81-4218-ac1c-c15c08bdb463 admin admin] PATH_INFO: `/v3/roles` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:20:39.437810 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd69c89-5b81-4218-ac1c-c15c08bdb463 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:20:39.439454 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd69c89-5b81-4218-ac1c-c15c08bdb463 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:20:39.449527 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1285/2547] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 10 12:20:39 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 12:20:39.453819 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5df070-0eec-4b56-a0cd-7dfe30b52768 admin admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:20:39.465627 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5df070-0eec-4b56-a0cd-7dfe30b52768 admin admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:20:39.470472 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0402a434-a452-4a33-a29b-374fb877673c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:20:39.471162 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0402a434-a452-4a33-a29b-374fb877673c admin admin] REQUEST_METHOD: `PUT` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:20:39.471447 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0402a434-a452-4a33-a29b-374fb877673c admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:20:39.471722 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0402a434-a452-4a33-a29b-374fb877673c admin admin] PATH_INFO: `/v3/projects/cb5dbfe7641b4b42b4aed0ca4dcaa808/users/219e27f423b64f2d95c4e3b163ea38ed/roles/0d822a0674b6495fac623877a3b38025` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:20:39.473689 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0402a434-a452-4a33-a29b-374fb877673c admin admin] RBAC: Authorizing `identity:create_grant(project_id=cb5dbfe7641b4b42b4aed0ca4dcaa808, user_id=219e27f423b64f2d95c4e3b163ea38ed, role_id=0d822a0674b6495fac623877a3b38025)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:20:39.476538 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0402a434-a452-4a33-a29b-374fb877673c admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:20:39.487528 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1263/2548] 10.222.0.46 () {60 vars in 1483 bytes} [Mon Oct 10 12:20:39 2022] PUT /identity/v3/projects/cb5dbfe7641b4b42b4aed0ca4dcaa808/users/219e27f423b64f2d95c4e3b163ea38ed/roles/0d822a0674b6495fac623877a3b38025 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:20:39.497665 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d917a7bd-a99e-4821-b72f-f4867bb4ba04 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:20:39.497958 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d917a7bd-a99e-4821-b72f-f4867bb4ba04 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:20:39.498245 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d917a7bd-a99e-4821-b72f-f4867bb4ba04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:20:39.528570 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-d917a7bd-a99e-4821-b72f-f4867bb4ba04 None None] MFA Rules not processed for user `219e27f423b64f2d95c4e3b163ea38ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:20:39.529801 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-d917a7bd-a99e-4821-b72f-f4867bb4ba04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:20:39.599138 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1286/2549] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:20:39 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:20:39.611272 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0402a434-a452-4a33-a29b-374fb877673c admin admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:20:39.612959 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-0402a434-a452-4a33-a29b-374fb877673c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:20:39.654710 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:20:39.671265 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0402a434-a452-4a33-a29b-374fb877673c admin admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:20:39.675044 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f0d91e-135c-4a80-becc-7a457ee2ef22 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3ae093470484b93b23dcfae1657df9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad3772f7b6a47c28b7ba37a9316a3c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:20:39.675649 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f0d91e-135c-4a80-becc-7a457ee2ef22 service cinder] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:20:39.675930 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f0d91e-135c-4a80-becc-7a457ee2ef22 service cinder] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:20:39.676204 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f0d91e-135c-4a80-becc-7a457ee2ef22 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:20:39.735289 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f0d91e-135c-4a80-becc-7a457ee2ef22 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:20:39.736716 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f0d91e-135c-4a80-becc-7a457ee2ef22 service cinder] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:20:39.746416 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1264/2550] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 10 12:20:39 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:20:42.785628 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d917a7bd-a99e-4821-b72f-f4867bb4ba04 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:20:42.795720 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d917a7bd-a99e-4821-b72f-f4867bb4ba04 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:20:42.799464 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c796250-0fcf-4a44-a28b-7ded6dc78aa4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3ae093470484b93b23dcfae1657df9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad3772f7b6a47c28b7ba37a9316a3c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:20:42.800053 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c796250-0fcf-4a44-a28b-7ded6dc78aa4 service cinder] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:20:42.800327 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c796250-0fcf-4a44-a28b-7ded6dc78aa4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:20:42.800596 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c796250-0fcf-4a44-a28b-7ded6dc78aa4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:20:42.863016 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c796250-0fcf-4a44-a28b-7ded6dc78aa4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:20:42.864462 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c796250-0fcf-4a44-a28b-7ded6dc78aa4 service cinder] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:20:42.874818 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1287/2551] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 10 12:20:42 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:20:44.577567 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f0d91e-135c-4a80-becc-7a457ee2ef22 service cinder] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:20:44.587806 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f0d91e-135c-4a80-becc-7a457ee2ef22 service cinder] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:20:44.591655 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9bcaae9-d316-40b0-97a7-a59736e08b33 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3ae093470484b93b23dcfae1657df9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad3772f7b6a47c28b7ba37a9316a3c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:20:44.592271 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bcaae9-d316-40b0-97a7-a59736e08b33 service cinder] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:20:44.592555 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bcaae9-d316-40b0-97a7-a59736e08b33 service cinder] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:20:44.592832 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bcaae9-d316-40b0-97a7-a59736e08b33 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:20:44.602466 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9bcaae9-d316-40b0-97a7-a59736e08b33 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:20:44.603984 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9bcaae9-d316-40b0-97a7-a59736e08b33 service cinder] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:20:44.614331 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1265/2552] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 10 12:20:44 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:20:51.101377 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c796250-0fcf-4a44-a28b-7ded6dc78aa4 service cinder] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:20:51.135893 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:20:51.171639 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c796250-0fcf-4a44-a28b-7ded6dc78aa4 service cinder] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:20:51.176618 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d916b067-1b33-47d6-83e1-d1702cb95866 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:20:51.177814 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d916b067-1b33-47d6-83e1-d1702cb95866 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:20:51.178167 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d916b067-1b33-47d6-83e1-d1702cb95866 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:20:51.178450 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d916b067-1b33-47d6-83e1-d1702cb95866 admin admin] PATH_INFO: `/v3/users/5e63cfd796074ca69f78e18aec2a4c85` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:20:51.179692 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d916b067-1b33-47d6-83e1-d1702cb95866 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5e63cfd796074ca69f78e18aec2a4c85)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:20:51.181676 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d916b067-1b33-47d6-83e1-d1702cb95866 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:20:51.415627 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-d916b067-1b33-47d6-83e1-d1702cb95866 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5e63cfd796074ca69f78e18aec2a4c85'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:20:51.423047 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-d916b067-1b33-47d6-83e1-d1702cb95866 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5e63cfd796074ca69f78e18aec2a4c85'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:20:51.423786 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-d916b067-1b33-47d6-83e1-d1702cb95866 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5e63cfd796074ca69f78e18aec2a4c85'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:20:51.525540 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-d916b067-1b33-47d6-83e1-d1702cb95866 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5e63cfd796074ca69f78e18aec2a4c85'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:20:51.540222 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-d916b067-1b33-47d6-83e1-d1702cb95866 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5e63cfd796074ca69f78e18aec2a4c85'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:20:51.547776 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1288/2553] 10.222.0.46 () {58 vars in 1224 bytes} [Mon Oct 10 12:20:51 2022] DELETE /identity/v3/users/5e63cfd796074ca69f78e18aec2a4c85 => generated 0 bytes in 447 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:20:51.553935 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804126a2-8a17-417f-9752-6da03e1594da None None] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:20:51.554707 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804126a2-8a17-417f-9752-6da03e1594da None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:20:51.554707 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804126a2-8a17-417f-9752-6da03e1594da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:20:51.576397 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.auth.core [None req-804126a2-8a17-417f-9752-6da03e1594da None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:20:51.577639 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-804126a2-8a17-417f-9752-6da03e1594da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:20:51.642628 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1266/2554] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:20:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:20:51.653391 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d916b067-1b33-47d6-83e1-d1702cb95866 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:20:51.655098 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-d916b067-1b33-47d6-83e1-d1702cb95866 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:20:51.725500 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d916b067-1b33-47d6-83e1-d1702cb95866 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:20:51.729221 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32065293-b158-4b22-98e2-02ffb9139ca7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58b1c7c85302421cba823ace4201861f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad3772f7b6a47c28b7ba37a9316a3c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:20:51.729847 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32065293-b158-4b22-98e2-02ffb9139ca7 service neutron] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:20:51.730129 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32065293-b158-4b22-98e2-02ffb9139ca7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:20:51.730409 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32065293-b158-4b22-98e2-02ffb9139ca7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:20:51.741673 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32065293-b158-4b22-98e2-02ffb9139ca7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:20:51.743187 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32065293-b158-4b22-98e2-02ffb9139ca7 service neutron] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:20:51.753908 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1289/2555] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 10 12:20:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:20:52.368415 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804126a2-8a17-417f-9752-6da03e1594da None None] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:20:52.369989 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-804126a2-8a17-417f-9752-6da03e1594da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:20:52.382762 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804126a2-8a17-417f-9752-6da03e1594da None None] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:20:52.387657 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c4d123-88c2-485c-8f72-9863a5a07362 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:20:52.388302 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c4d123-88c2-485c-8f72-9863a5a07362 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:20:52.388575 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c4d123-88c2-485c-8f72-9863a5a07362 admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:20:52.388852 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c4d123-88c2-485c-8f72-9863a5a07362 admin admin] PATH_INFO: `/v3/projects/93fe89d358aa44afb40f67f46596c571` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:20:52.390018 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c4d123-88c2-485c-8f72-9863a5a07362 admin admin] RBAC: Authorizing `identity:delete_project(project_id=93fe89d358aa44afb40f67f46596c571)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:20:52.392145 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c4d123-88c2-485c-8f72-9863a5a07362 admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:20:52.568250 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-a0c4d123-88c2-485c-8f72-9863a5a07362 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '93fe89d358aa44afb40f67f46596c571', 'request_id': 'req-a0c4d123-88c2-485c-8f72-9863a5a07362'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:20:52.578482 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-a0c4d123-88c2-485c-8f72-9863a5a07362 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '93fe89d358aa44afb40f67f46596c571', 'request_id': 'req-a0c4d123-88c2-485c-8f72-9863a5a07362'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:20:52.589980 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-a0c4d123-88c2-485c-8f72-9863a5a07362 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '93fe89d358aa44afb40f67f46596c571', 'request_id': 'req-a0c4d123-88c2-485c-8f72-9863a5a07362'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:20:52.660045 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:20:52.664647 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-a0c4d123-88c2-485c-8f72-9863a5a07362 admin admin] The token cache is being invalidate because project 93fe89d358aa44afb40f67f46596c571 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81676) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 10 12:20:52.664935 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-a0c4d123-88c2-485c-8f72-9863a5a07362 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:20:52.665580 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-a0c4d123-88c2-485c-8f72-9863a5a07362 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:20:52.667231 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1267/2556] 10.222.0.46 () {58 vars in 1233 bytes} [Mon Oct 10 12:20:52 2022] DELETE /identity/v3/projects/93fe89d358aa44afb40f67f46596c571 => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:20:52.671345 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32065293-b158-4b22-98e2-02ffb9139ca7 service neutron] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:20:52.681685 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32065293-b158-4b22-98e2-02ffb9139ca7 service neutron] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:20:52.729230 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:20:52.741159 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2304381-d25c-4eae-9200-cb6d1effaf74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:20:52.741801 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2304381-d25c-4eae-9200-cb6d1effaf74 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:20:52.742085 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2304381-d25c-4eae-9200-cb6d1effaf74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:20:52.742364 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2304381-d25c-4eae-9200-cb6d1effaf74 admin admin] PATH_INFO: `/v3/users/8dd1d4c3b9844dc5b3c1d7e3407547b2` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:20:52.743714 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2304381-d25c-4eae-9200-cb6d1effaf74 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8dd1d4c3b9844dc5b3c1d7e3407547b2)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:20:52.745666 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2304381-d25c-4eae-9200-cb6d1effaf74 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:20:52.963649 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-e2304381-d25c-4eae-9200-cb6d1effaf74 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8dd1d4c3b9844dc5b3c1d7e3407547b2'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:20:52.971305 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-e2304381-d25c-4eae-9200-cb6d1effaf74 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8dd1d4c3b9844dc5b3c1d7e3407547b2'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:20:52.972113 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-e2304381-d25c-4eae-9200-cb6d1effaf74 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8dd1d4c3b9844dc5b3c1d7e3407547b2'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:20:53.045527 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-e2304381-d25c-4eae-9200-cb6d1effaf74 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8dd1d4c3b9844dc5b3c1d7e3407547b2'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:20:53.060996 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-e2304381-d25c-4eae-9200-cb6d1effaf74 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8dd1d4c3b9844dc5b3c1d7e3407547b2'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:20:53.068223 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1290/2557] 10.222.0.46 () {58 vars in 1224 bytes} [Mon Oct 10 12:20:52 2022] DELETE /identity/v3/users/8dd1d4c3b9844dc5b3c1d7e3407547b2 => generated 0 bytes in 397 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:20:53.529169 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c4d123-88c2-485c-8f72-9863a5a07362 admin admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:20:53.531148 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-a0c4d123-88c2-485c-8f72-9863a5a07362 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:20:53.546819 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c4d123-88c2-485c-8f72-9863a5a07362 admin admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:20:53.648817 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-affe87be-09e4-426a-8211-d4265a33f3ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:20:53.649892 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affe87be-09e4-426a-8211-d4265a33f3ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:20:53.650331 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affe87be-09e4-426a-8211-d4265a33f3ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:20:53.650851 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affe87be-09e4-426a-8211-d4265a33f3ed admin admin] PATH_INFO: `/v3/projects/db1fef7ea1c74d47b44e6ba7ce27f7de` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:20:53.652372 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-affe87be-09e4-426a-8211-d4265a33f3ed admin admin] RBAC: Authorizing `identity:delete_project(project_id=db1fef7ea1c74d47b44e6ba7ce27f7de)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:20:53.655343 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-affe87be-09e4-426a-8211-d4265a33f3ed admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:20:53.775992 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-affe87be-09e4-426a-8211-d4265a33f3ed admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'db1fef7ea1c74d47b44e6ba7ce27f7de', 'request_id': 'req-affe87be-09e4-426a-8211-d4265a33f3ed'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:20:53.782130 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-affe87be-09e4-426a-8211-d4265a33f3ed admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'db1fef7ea1c74d47b44e6ba7ce27f7de', 'request_id': 'req-affe87be-09e4-426a-8211-d4265a33f3ed'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:20:53.788866 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-affe87be-09e4-426a-8211-d4265a33f3ed admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'db1fef7ea1c74d47b44e6ba7ce27f7de', 'request_id': 'req-affe87be-09e4-426a-8211-d4265a33f3ed'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:20:53.879945 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-affe87be-09e4-426a-8211-d4265a33f3ed admin admin] The token cache is being invalidate because project db1fef7ea1c74d47b44e6ba7ce27f7de was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81676) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 10 12:20:53.880229 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-affe87be-09e4-426a-8211-d4265a33f3ed admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:20:53.881603 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-affe87be-09e4-426a-8211-d4265a33f3ed admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:20:53.883533 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1268/2558] 10.222.0.46 () {58 vars in 1233 bytes} [Mon Oct 10 12:20:53 2022] DELETE /identity/v3/projects/db1fef7ea1c74d47b44e6ba7ce27f7de => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:20:53.888331 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2304381-d25c-4eae-9200-cb6d1effaf74 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:20:53.889991 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-e2304381-d25c-4eae-9200-cb6d1effaf74 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:20:53.903507 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2304381-d25c-4eae-9200-cb6d1effaf74 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:20:53.964611 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ec059f-4559-4322-b25f-5b9c0b8a19b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:20:53.965262 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ec059f-4559-4322-b25f-5b9c0b8a19b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:20:53.965544 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ec059f-4559-4322-b25f-5b9c0b8a19b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:20:53.965824 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ec059f-4559-4322-b25f-5b9c0b8a19b0 admin admin] PATH_INFO: `/v3/users/219e27f423b64f2d95c4e3b163ea38ed` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:20:53.967196 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ec059f-4559-4322-b25f-5b9c0b8a19b0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=219e27f423b64f2d95c4e3b163ea38ed)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:20:53.969197 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ec059f-4559-4322-b25f-5b9c0b8a19b0 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:20:54.169922 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-70ec059f-4559-4322-b25f-5b9c0b8a19b0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '219e27f423b64f2d95c4e3b163ea38ed'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:20:54.179846 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-70ec059f-4559-4322-b25f-5b9c0b8a19b0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '219e27f423b64f2d95c4e3b163ea38ed'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:20:54.180584 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-70ec059f-4559-4322-b25f-5b9c0b8a19b0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '219e27f423b64f2d95c4e3b163ea38ed'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:20:54.249231 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-70ec059f-4559-4322-b25f-5b9c0b8a19b0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '219e27f423b64f2d95c4e3b163ea38ed'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:20:54.256098 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:20:54.265121 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-70ec059f-4559-4322-b25f-5b9c0b8a19b0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '219e27f423b64f2d95c4e3b163ea38ed'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:20:54.271888 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1291/2559] 10.222.0.46 () {58 vars in 1224 bytes} [Mon Oct 10 12:20:53 2022] DELETE /identity/v3/users/219e27f423b64f2d95c4e3b163ea38ed => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:20:54.685570 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-affe87be-09e4-426a-8211-d4265a33f3ed admin admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:20:54.687123 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-affe87be-09e4-426a-8211-d4265a33f3ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:20:54.700820 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-affe87be-09e4-426a-8211-d4265a33f3ed admin admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:20:54.765801 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7837cfca-5ad7-42bd-9e1c-9a8888ca2e8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:20:54.766449 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7837cfca-5ad7-42bd-9e1c-9a8888ca2e8d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:20:54.766796 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7837cfca-5ad7-42bd-9e1c-9a8888ca2e8d admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:20:54.767030 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7837cfca-5ad7-42bd-9e1c-9a8888ca2e8d admin admin] PATH_INFO: `/v3/projects/cb5dbfe7641b4b42b4aed0ca4dcaa808` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:20:54.768262 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7837cfca-5ad7-42bd-9e1c-9a8888ca2e8d admin admin] RBAC: Authorizing `identity:delete_project(project_id=cb5dbfe7641b4b42b4aed0ca4dcaa808)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:20:54.770208 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7837cfca-5ad7-42bd-9e1c-9a8888ca2e8d admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:20:54.868863 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:20:54.885106 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-7837cfca-5ad7-42bd-9e1c-9a8888ca2e8d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cb5dbfe7641b4b42b4aed0ca4dcaa808', 'request_id': 'req-7837cfca-5ad7-42bd-9e1c-9a8888ca2e8d'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:20:54.893459 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-7837cfca-5ad7-42bd-9e1c-9a8888ca2e8d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cb5dbfe7641b4b42b4aed0ca4dcaa808', 'request_id': 'req-7837cfca-5ad7-42bd-9e1c-9a8888ca2e8d'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:20:54.901804 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-7837cfca-5ad7-42bd-9e1c-9a8888ca2e8d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cb5dbfe7641b4b42b4aed0ca4dcaa808', 'request_id': 'req-7837cfca-5ad7-42bd-9e1c-9a8888ca2e8d'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:20:54.987153 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-7837cfca-5ad7-42bd-9e1c-9a8888ca2e8d admin admin] The token cache is being invalidate because project cb5dbfe7641b4b42b4aed0ca4dcaa808 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81676) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 10 12:20:54.987440 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-7837cfca-5ad7-42bd-9e1c-9a8888ca2e8d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:20:54.988165 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-7837cfca-5ad7-42bd-9e1c-9a8888ca2e8d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:20:54.989772 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1269/2560] 10.222.0.46 () {58 vars in 1233 bytes} [Mon Oct 10 12:20:54 2022] DELETE /identity/v3/projects/cb5dbfe7641b4b42b4aed0ca4dcaa808 => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:20:54.995024 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9317f048-e490-4c88-bd8d-d156ff7b7212 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:20:54.995616 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9317f048-e490-4c88-bd8d-d156ff7b7212 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:20:54.995616 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9317f048-e490-4c88-bd8d-d156ff7b7212 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:20:55.018161 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-9317f048-e490-4c88-bd8d-d156ff7b7212 None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:20:55.019522 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-9317f048-e490-4c88-bd8d-d156ff7b7212 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:20:55.095744 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1292/2561] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:20:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:20:55.104368 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195592d4-4594-44ff-8ac9-4b314cad19ae None None] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:20:55.108805 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195592d4-4594-44ff-8ac9-4b314cad19ae None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:20:55.108805 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195592d4-4594-44ff-8ac9-4b314cad19ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:20:55.127942 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.auth.core [None req-195592d4-4594-44ff-8ac9-4b314cad19ae None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:20:55.129480 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-195592d4-4594-44ff-8ac9-4b314cad19ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:20:55.139962 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1270/2562] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:20:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:20:55.150139 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9317f048-e490-4c88-bd8d-d156ff7b7212 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:20:55.156420 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9317f048-e490-4c88-bd8d-d156ff7b7212 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:20:55.162275 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856ff878-47fc-478b-a55c-da8d8805ab01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:20:55.162955 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856ff878-47fc-478b-a55c-da8d8805ab01 admin admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:20:55.163271 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856ff878-47fc-478b-a55c-da8d8805ab01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:20:55.163516 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856ff878-47fc-478b-a55c-da8d8805ab01 admin admin] PATH_INFO: `/v3/domains` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:20:55.164512 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856ff878-47fc-478b-a55c-da8d8805ab01 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 12:20:55.164512 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856ff878-47fc-478b-a55c-da8d8805ab01 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:20:55.166150 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856ff878-47fc-478b-a55c-da8d8805ab01 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:20:55.186276 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1293/2563] 10.222.0.46 () {58 vars in 1153 bytes} [Mon Oct 10 12:20:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 12:20:55.190871 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195592d4-4594-44ff-8ac9-4b314cad19ae None None] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:20:55.201391 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195592d4-4594-44ff-8ac9-4b314cad19ae None None] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:20:55.206471 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66feb6ff-e58d-4433-b861-a63e91671f5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:20:55.207111 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66feb6ff-e58d-4433-b861-a63e91671f5c admin admin] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:20:55.207390 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66feb6ff-e58d-4433-b861-a63e91671f5c admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:20:55.207662 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66feb6ff-e58d-4433-b861-a63e91671f5c admin admin] PATH_INFO: `/v3/projects` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:20:55.208929 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66feb6ff-e58d-4433-b861-a63e91671f5c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:20:55.210838 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66feb6ff-e58d-4433-b861-a63e91671f5c admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:20:55.532868 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1271/2564] 10.222.0.46 () {60 vars in 1153 bytes} [Mon Oct 10 12:20:55 2022] POST /identity/v3/projects => generated 369 bytes in 342 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 12:20:55.537517 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856ff878-47fc-478b-a55c-da8d8805ab01 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:20:55.548039 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856ff878-47fc-478b-a55c-da8d8805ab01 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:20:55.616672 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ef667f-68e5-45a0-b1d8-3395ac60240d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:20:55.617297 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ef667f-68e5-45a0-b1d8-3395ac60240d admin admin] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:20:55.617581 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ef667f-68e5-45a0-b1d8-3395ac60240d admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:20:55.617853 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ef667f-68e5-45a0-b1d8-3395ac60240d admin admin] PATH_INFO: `/v3/users` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:20:55.618488 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ef667f-68e5-45a0-b1d8-3395ac60240d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:20:55.620448 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ef667f-68e5-45a0-b1d8-3395ac60240d admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:20:55.727086 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1294/2565] 10.222.0.46 () {60 vars in 1144 bytes} [Mon Oct 10 12:20:55 2022] POST /identity/v3/users => generated 285 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 12:20:55.732240 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66feb6ff-e58d-4433-b861-a63e91671f5c admin admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:20:55.752119 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66feb6ff-e58d-4433-b861-a63e91671f5c admin admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:20:55.752119 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ccbc4a-e9ca-4da7-8964-430e966a85e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:20:55.752119 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ccbc4a-e9ca-4da7-8964-430e966a85e1 admin admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:20:55.752119 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ccbc4a-e9ca-4da7-8964-430e966a85e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:20:55.752662 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ccbc4a-e9ca-4da7-8964-430e966a85e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:20:55.752662 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ccbc4a-e9ca-4da7-8964-430e966a85e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:20:55.752662 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ccbc4a-e9ca-4da7-8964-430e966a85e1 admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:20:55.763122 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1272/2566] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 10 12:20:55 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 12:20:55.769239 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ef667f-68e5-45a0-b1d8-3395ac60240d admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:20:55.785690 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ef667f-68e5-45a0-b1d8-3395ac60240d admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:20:55.791538 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5176b8a8-254a-43e0-b682-349d063fe906 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:20:55.792169 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5176b8a8-254a-43e0-b682-349d063fe906 admin admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:20:55.792448 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5176b8a8-254a-43e0-b682-349d063fe906 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:20:55.792725 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5176b8a8-254a-43e0-b682-349d063fe906 admin admin] PATH_INFO: `/v3/roles` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:20:55.793383 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5176b8a8-254a-43e0-b682-349d063fe906 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:20:55.795053 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5176b8a8-254a-43e0-b682-349d063fe906 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:20:55.807964 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1295/2567] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 10 12:20:55 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 12:20:55.812486 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ccbc4a-e9ca-4da7-8964-430e966a85e1 admin admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:20:55.822984 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ccbc4a-e9ca-4da7-8964-430e966a85e1 admin admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:20:55.828418 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7116c3e8-4b4b-4e6c-88b3-5af76edf8f72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:20:55.829163 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7116c3e8-4b4b-4e6c-88b3-5af76edf8f72 admin admin] REQUEST_METHOD: `PUT` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:20:55.829476 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7116c3e8-4b4b-4e6c-88b3-5af76edf8f72 admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:20:55.829791 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7116c3e8-4b4b-4e6c-88b3-5af76edf8f72 admin admin] PATH_INFO: `/v3/projects/a037f5e5d96a48cb800baea601c9fb0f/users/eb72ac91a0b848c1914239e868e2a445/roles/b2cb2a6e818e47338ce071e6fedfc673` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:20:55.846237 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7116c3e8-4b4b-4e6c-88b3-5af76edf8f72 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a037f5e5d96a48cb800baea601c9fb0f, user_id=eb72ac91a0b848c1914239e868e2a445, role_id=b2cb2a6e818e47338ce071e6fedfc673)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:20:55.849491 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7116c3e8-4b4b-4e6c-88b3-5af76edf8f72 admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:20:55.946921 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1273/2568] 10.222.0.46 () {60 vars in 1483 bytes} [Mon Oct 10 12:20:55 2022] PUT /identity/v3/projects/a037f5e5d96a48cb800baea601c9fb0f/users/eb72ac91a0b848c1914239e868e2a445/roles/b2cb2a6e818e47338ce071e6fedfc673 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:20:55.958319 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a677024c-dc31-4feb-b532-9e92d2892b53 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:20:55.958750 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a677024c-dc31-4feb-b532-9e92d2892b53 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:20:55.959079 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a677024c-dc31-4feb-b532-9e92d2892b53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:20:55.991818 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-a677024c-dc31-4feb-b532-9e92d2892b53 None None] MFA Rules not processed for user `eb72ac91a0b848c1914239e868e2a445`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:20:55.993056 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-a677024c-dc31-4feb-b532-9e92d2892b53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:20:56.039848 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:20:56.074262 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1296/2569] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:20:55 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:20:56.088621 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7116c3e8-4b4b-4e6c-88b3-5af76edf8f72 admin admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:20:56.090662 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-7116c3e8-4b4b-4e6c-88b3-5af76edf8f72 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:20:56.156390 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7116c3e8-4b4b-4e6c-88b3-5af76edf8f72 admin admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:20:56.160737 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1dcde4f-6af5-489d-b7db-cef278ab48c7 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '124f4aa8ab244ca2aadf4f2614df51e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad3772f7b6a47c28b7ba37a9316a3c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:20:56.161375 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1dcde4f-6af5-489d-b7db-cef278ab48c7 service glance] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:20:56.161656 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1dcde4f-6af5-489d-b7db-cef278ab48c7 service glance] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:20:56.161936 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1dcde4f-6af5-489d-b7db-cef278ab48c7 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:20:56.172005 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1dcde4f-6af5-489d-b7db-cef278ab48c7 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:20:56.173469 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1dcde4f-6af5-489d-b7db-cef278ab48c7 service glance] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:20:56.184343 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1274/2570] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 10 12:20:56 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:21:06.128806 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=8 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:21:06.179569 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=65,keystone:DELETE=5,keystone:INSERT=3 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:21:21.667459 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a677024c-dc31-4feb-b532-9e92d2892b53 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:21:21.669046 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-a677024c-dc31-4feb-b532-9e92d2892b53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:21:21.766661 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a677024c-dc31-4feb-b532-9e92d2892b53 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:21:21.772261 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ac79b0-37a9-40a4-8432-3a8ddef65be5 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e8f9bb1776d47169ef7febb57c954cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad3772f7b6a47c28b7ba37a9316a3c1', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:21:21.773360 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ac79b0-37a9-40a4-8432-3a8ddef65be5 service placement] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:21:21.773644 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ac79b0-37a9-40a4-8432-3a8ddef65be5 service placement] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:21:21.773919 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ac79b0-37a9-40a4-8432-3a8ddef65be5 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:21:21.776289 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-68ac79b0-37a9-40a4-8432-3a8ddef65be5 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:21:21.791008 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ac79b0-37a9-40a4-8432-3a8ddef65be5 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:21:21.792999 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ac79b0-37a9-40a4-8432-3a8ddef65be5 service placement] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:21:21.813118 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1297/2571] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 10 12:21:21 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:21:31.807673 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=32 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:21:34.056129 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1dcde4f-6af5-489d-b7db-cef278ab48c7 service glance] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:21:34.057526 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-f1dcde4f-6af5-489d-b7db-cef278ab48c7 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:21:34.121418 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1dcde4f-6af5-489d-b7db-cef278ab48c7 service glance] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:21:34.125427 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebe93c1-a7c9-4939-8c06-c4dc2c5a4e5c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58b1c7c85302421cba823ace4201861f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad3772f7b6a47c28b7ba37a9316a3c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:21:34.126034 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebe93c1-a7c9-4939-8c06-c4dc2c5a4e5c service neutron] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:21:34.126326 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebe93c1-a7c9-4939-8c06-c4dc2c5a4e5c service neutron] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:21:34.126633 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebe93c1-a7c9-4939-8c06-c4dc2c5a4e5c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:21:34.128517 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-cebe93c1-a7c9-4939-8c06-c4dc2c5a4e5c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:21:34.192595 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cebe93c1-a7c9-4939-8c06-c4dc2c5a4e5c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:21:34.194037 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cebe93c1-a7c9-4939-8c06-c4dc2c5a4e5c service neutron] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:21:34.204100 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1275/2572] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 10 12:21:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:21:35.793910 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ac79b0-37a9-40a4-8432-3a8ddef65be5 service placement] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:21:35.795597 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-68ac79b0-37a9-40a4-8432-3a8ddef65be5 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:21:35.817857 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ac79b0-37a9-40a4-8432-3a8ddef65be5 service placement] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:21:35.819787 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710e53f5-4e43-4af8-b10d-3f2d5b360dad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58b1c7c85302421cba823ace4201861f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad3772f7b6a47c28b7ba37a9316a3c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:21:35.820409 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710e53f5-4e43-4af8-b10d-3f2d5b360dad service neutron] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:21:35.820879 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710e53f5-4e43-4af8-b10d-3f2d5b360dad service neutron] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:21:35.821329 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710e53f5-4e43-4af8-b10d-3f2d5b360dad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:21:35.823556 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-710e53f5-4e43-4af8-b10d-3f2d5b360dad service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:21:35.839356 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710e53f5-4e43-4af8-b10d-3f2d5b360dad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:21:35.841119 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710e53f5-4e43-4af8-b10d-3f2d5b360dad service neutron] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:21:35.858777 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1298/2573] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 10 12:21:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:21:37.444546 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebe93c1-a7c9-4939-8c06-c4dc2c5a4e5c service neutron] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:21:37.454530 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebe93c1-a7c9-4939-8c06-c4dc2c5a4e5c service neutron] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:21:37.464336 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824c8327-1d33-4eed-b797-637e55a2fc83 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58b1c7c85302421cba823ace4201861f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad3772f7b6a47c28b7ba37a9316a3c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:21:37.464336 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824c8327-1d33-4eed-b797-637e55a2fc83 service neutron] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:21:37.464336 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824c8327-1d33-4eed-b797-637e55a2fc83 service neutron] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:21:37.464336 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824c8327-1d33-4eed-b797-637e55a2fc83 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:21:37.464707 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-824c8327-1d33-4eed-b797-637e55a2fc83 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:21:37.481194 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824c8327-1d33-4eed-b797-637e55a2fc83 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:21:37.481474 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824c8327-1d33-4eed-b797-637e55a2fc83 service neutron] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:21:37.495820 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1276/2574] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 10 12:21:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:21:39.515247 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710e53f5-4e43-4af8-b10d-3f2d5b360dad service neutron] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:21:39.517811 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-710e53f5-4e43-4af8-b10d-3f2d5b360dad service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:21:39.536575 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710e53f5-4e43-4af8-b10d-3f2d5b360dad service neutron] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:21:39.542288 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725993bb-ff79-494b-ada7-42322b427667 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:21:39.543241 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725993bb-ff79-494b-ada7-42322b427667 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:21:39.543521 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725993bb-ff79-494b-ada7-42322b427667 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:21:39.543855 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725993bb-ff79-494b-ada7-42322b427667 admin admin] PATH_INFO: `/v3/users/a7a87488f78444c3ad820527e5fbc099` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:21:39.544979 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725993bb-ff79-494b-ada7-42322b427667 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a7a87488f78444c3ad820527e5fbc099)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:21:39.546932 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725993bb-ff79-494b-ada7-42322b427667 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:21:39.808482 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-725993bb-ff79-494b-ada7-42322b427667 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a7a87488f78444c3ad820527e5fbc099'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:21:39.819038 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-725993bb-ff79-494b-ada7-42322b427667 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a7a87488f78444c3ad820527e5fbc099'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:21:39.820950 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-725993bb-ff79-494b-ada7-42322b427667 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a7a87488f78444c3ad820527e5fbc099'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:21:39.930488 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-725993bb-ff79-494b-ada7-42322b427667 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a7a87488f78444c3ad820527e5fbc099'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:21:39.963268 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-725993bb-ff79-494b-ada7-42322b427667 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a7a87488f78444c3ad820527e5fbc099'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:21:39.974279 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1299/2575] 10.222.0.46 () {58 vars in 1224 bytes} [Mon Oct 10 12:21:39 2022] DELETE /identity/v3/users/a7a87488f78444c3ad820527e5fbc099 => generated 0 bytes in 460 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:21:40.501632 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824c8327-1d33-4eed-b797-637e55a2fc83 service neutron] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:21:40.504386 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725993bb-ff79-494b-ada7-42322b427667 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:21:40.506874 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-725993bb-ff79-494b-ada7-42322b427667 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:21:40.525204 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824c8327-1d33-4eed-b797-637e55a2fc83 service neutron] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:21:40.635815 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725993bb-ff79-494b-ada7-42322b427667 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:21:40.636640 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bd7f0a-9734-4fe8-8809-fa380aa517f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:21:40.637435 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bd7f0a-9734-4fe8-8809-fa380aa517f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:21:40.637832 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bd7f0a-9734-4fe8-8809-fa380aa517f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:21:40.638238 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bd7f0a-9734-4fe8-8809-fa380aa517f5 admin admin] PATH_INFO: `/v3/projects/af22e9604d79499aa9b7c2ee3cef156d` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:21:40.639549 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5bd7f0a-9734-4fe8-8809-fa380aa517f5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=af22e9604d79499aa9b7c2ee3cef156d)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:21:40.641582 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5bd7f0a-9734-4fe8-8809-fa380aa517f5 admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:21:40.642292 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45fdd79-1ef8-4149-85dd-de577d7ab523 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e8f9bb1776d47169ef7febb57c954cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad3772f7b6a47c28b7ba37a9316a3c1', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:21:40.645132 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45fdd79-1ef8-4149-85dd-de577d7ab523 service placement] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:21:40.645991 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45fdd79-1ef8-4149-85dd-de577d7ab523 service placement] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:21:40.647409 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45fdd79-1ef8-4149-85dd-de577d7ab523 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:21:40.649844 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-b45fdd79-1ef8-4149-85dd-de577d7ab523 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:21:40.672462 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45fdd79-1ef8-4149-85dd-de577d7ab523 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:21:40.674952 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45fdd79-1ef8-4149-85dd-de577d7ab523 service placement] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:21:40.686487 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1300/2576] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 10 12:21:40 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 10 12:21:40.774193 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:21:40.781598 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-d5bd7f0a-9734-4fe8-8809-fa380aa517f5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'af22e9604d79499aa9b7c2ee3cef156d', 'request_id': 'req-d5bd7f0a-9734-4fe8-8809-fa380aa517f5'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:21:40.787259 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-d5bd7f0a-9734-4fe8-8809-fa380aa517f5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'af22e9604d79499aa9b7c2ee3cef156d', 'request_id': 'req-d5bd7f0a-9734-4fe8-8809-fa380aa517f5'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:21:40.793586 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-d5bd7f0a-9734-4fe8-8809-fa380aa517f5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'af22e9604d79499aa9b7c2ee3cef156d', 'request_id': 'req-d5bd7f0a-9734-4fe8-8809-fa380aa517f5'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:21:40.889940 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-d5bd7f0a-9734-4fe8-8809-fa380aa517f5 admin admin] The token cache is being invalidate because project af22e9604d79499aa9b7c2ee3cef156d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81676) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 10 12:21:40.890240 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-d5bd7f0a-9734-4fe8-8809-fa380aa517f5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:21:40.890914 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-d5bd7f0a-9734-4fe8-8809-fa380aa517f5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:21:40.892445 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1277/2577] 10.222.0.46 () {58 vars in 1233 bytes} [Mon Oct 10 12:21:40 2022] DELETE /identity/v3/projects/af22e9604d79499aa9b7c2ee3cef156d => generated 0 bytes in 394 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:21:40.897477 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a5fc8c-c540-4ec7-a8ba-738fda9a9563 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:21:40.897775 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a5fc8c-c540-4ec7-a8ba-738fda9a9563 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:21:40.900542 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a5fc8c-c540-4ec7-a8ba-738fda9a9563 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:21:40.918201 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-59a5fc8c-c540-4ec7-a8ba-738fda9a9563 None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:21:40.919557 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-59a5fc8c-c540-4ec7-a8ba-738fda9a9563 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:21:40.969355 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:21:40.986597 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1301/2578] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:21:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 10 12:21:40.994863 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcd91a9-61be-4364-ad3f-a788c43a1f56 None None] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:21:40.995150 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcd91a9-61be-4364-ad3f-a788c43a1f56 None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:21:40.995458 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcd91a9-61be-4364-ad3f-a788c43a1f56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:21:41.016903 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.auth.core [None req-4fcd91a9-61be-4364-ad3f-a788c43a1f56 None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:21:41.018166 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-4fcd91a9-61be-4364-ad3f-a788c43a1f56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:21:41.028214 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1278/2579] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:21:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:21:41.042560 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a5fc8c-c540-4ec7-a8ba-738fda9a9563 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:21:41.059055 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a5fc8c-c540-4ec7-a8ba-738fda9a9563 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:21:41.065362 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7826fa86-d0b9-4b0b-8ba8-232c7d56b298 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:21:41.066003 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7826fa86-d0b9-4b0b-8ba8-232c7d56b298 admin admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:21:41.075441 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7826fa86-d0b9-4b0b-8ba8-232c7d56b298 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:21:41.075441 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7826fa86-d0b9-4b0b-8ba8-232c7d56b298 admin admin] PATH_INFO: `/v3/domains` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:21:41.075441 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7826fa86-d0b9-4b0b-8ba8-232c7d56b298 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 12:21:41.075441 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7826fa86-d0b9-4b0b-8ba8-232c7d56b298 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:21:41.075441 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7826fa86-d0b9-4b0b-8ba8-232c7d56b298 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:21:41.101898 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fcd91a9-61be-4364-ad3f-a788c43a1f56 None None] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:21:41.107438 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1302/2580] 10.222.0.46 () {58 vars in 1153 bytes} [Mon Oct 10 12:21:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 12:21:41.115539 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fcd91a9-61be-4364-ad3f-a788c43a1f56 None None] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:21:41.122648 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4a85ed-f7ad-48c6-a970-38264d84bc45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:21:41.122648 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4a85ed-f7ad-48c6-a970-38264d84bc45 admin admin] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:21:41.122648 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4a85ed-f7ad-48c6-a970-38264d84bc45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:21:41.122938 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4a85ed-f7ad-48c6-a970-38264d84bc45 admin admin] PATH_INFO: `/v3/projects` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:21:41.123617 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4a85ed-f7ad-48c6-a970-38264d84bc45 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:21:41.125735 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4a85ed-f7ad-48c6-a970-38264d84bc45 admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:21:41.234733 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1279/2581] 10.222.0.46 () {60 vars in 1153 bytes} [Mon Oct 10 12:21:41 2022] POST /identity/v3/projects => generated 365 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 12:21:41.237399 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7826fa86-d0b9-4b0b-8ba8-232c7d56b298 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:21:41.249737 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7826fa86-d0b9-4b0b-8ba8-232c7d56b298 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:21:41.368319 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33a56cc-ddaa-4247-9155-0438508ca3d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:21:41.369036 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33a56cc-ddaa-4247-9155-0438508ca3d0 admin admin] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:21:41.369319 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33a56cc-ddaa-4247-9155-0438508ca3d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:21:41.369577 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33a56cc-ddaa-4247-9155-0438508ca3d0 admin admin] PATH_INFO: `/v3/users` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:21:41.370213 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33a56cc-ddaa-4247-9155-0438508ca3d0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:21:41.372260 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33a56cc-ddaa-4247-9155-0438508ca3d0 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:21:41.514871 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1303/2582] 10.222.0.46 () {60 vars in 1144 bytes} [Mon Oct 10 12:21:41 2022] POST /identity/v3/users => generated 283 bytes in 278 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 10 12:21:41.519456 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4a85ed-f7ad-48c6-a970-38264d84bc45 admin admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:21:41.530179 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4a85ed-f7ad-48c6-a970-38264d84bc45 admin admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:21:41.536211 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd274c61-e99b-4bdb-bd23-4fd4c085cdf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:21:41.536936 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd274c61-e99b-4bdb-bd23-4fd4c085cdf7 admin admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:21:41.537319 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd274c61-e99b-4bdb-bd23-4fd4c085cdf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:21:41.537597 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd274c61-e99b-4bdb-bd23-4fd4c085cdf7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:21:41.538660 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd274c61-e99b-4bdb-bd23-4fd4c085cdf7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:21:41.565792 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd274c61-e99b-4bdb-bd23-4fd4c085cdf7 admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:21:41.565792 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1280/2583] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 10 12:21:41 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 10 12:21:41.567245 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33a56cc-ddaa-4247-9155-0438508ca3d0 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:21:41.582997 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33a56cc-ddaa-4247-9155-0438508ca3d0 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:21:41.589924 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e2d89d-4de3-4ffd-bb33-524ab3db264d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:21:41.593612 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e2d89d-4de3-4ffd-bb33-524ab3db264d admin admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:21:41.594747 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e2d89d-4de3-4ffd-bb33-524ab3db264d admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:21:41.595720 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e2d89d-4de3-4ffd-bb33-524ab3db264d admin admin] PATH_INFO: `/v3/roles` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:21:41.596744 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e2d89d-4de3-4ffd-bb33-524ab3db264d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:21:41.599759 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e2d89d-4de3-4ffd-bb33-524ab3db264d admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:21:41.625739 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1304/2584] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 10 12:21:41 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 12:21:41.630586 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd274c61-e99b-4bdb-bd23-4fd4c085cdf7 admin admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:21:41.660490 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd274c61-e99b-4bdb-bd23-4fd4c085cdf7 admin admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:21:41.682433 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26eabb78-2837-4d06-bb15-7fa0ed423e33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:21:41.683399 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26eabb78-2837-4d06-bb15-7fa0ed423e33 admin admin] REQUEST_METHOD: `PUT` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:21:41.683828 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26eabb78-2837-4d06-bb15-7fa0ed423e33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:21:41.684394 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26eabb78-2837-4d06-bb15-7fa0ed423e33 admin admin] PATH_INFO: `/v3/projects/ac4567ab040f46929f2d39006b0a9d5a/users/5185828d5a1149369fce35efd4c3e8b4/roles/b2cb2a6e818e47338ce071e6fedfc673` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:21:41.706888 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26eabb78-2837-4d06-bb15-7fa0ed423e33 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ac4567ab040f46929f2d39006b0a9d5a, user_id=5185828d5a1149369fce35efd4c3e8b4, role_id=b2cb2a6e818e47338ce071e6fedfc673)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:21:41.709931 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26eabb78-2837-4d06-bb15-7fa0ed423e33 admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:21:41.811823 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1281/2585] 10.222.0.46 () {60 vars in 1483 bytes} [Mon Oct 10 12:21:41 2022] PUT /identity/v3/projects/ac4567ab040f46929f2d39006b0a9d5a/users/5185828d5a1149369fce35efd4c3e8b4/roles/b2cb2a6e818e47338ce071e6fedfc673 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 10 12:21:41.822498 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165593fc-b812-455c-aa03-5c4f49bc6839 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:21:41.823551 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165593fc-b812-455c-aa03-5c4f49bc6839 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:21:41.823877 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165593fc-b812-455c-aa03-5c4f49bc6839 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:21:41.871503 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-165593fc-b812-455c-aa03-5c4f49bc6839 None None] MFA Rules not processed for user `5185828d5a1149369fce35efd4c3e8b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:21:41.872884 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-165593fc-b812-455c-aa03-5c4f49bc6839 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:21:42.017287 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26eabb78-2837-4d06-bb15-7fa0ed423e33 admin admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:21:42.019267 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1305/2586] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:21:41 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:21:42.145428 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26eabb78-2837-4d06-bb15-7fa0ed423e33 admin admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:21:42.152204 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a396fbab-7d80-43fd-acc4-8dd9dfe527ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:21:42.152956 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a396fbab-7d80-43fd-acc4-8dd9dfe527ef admin admin] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:21:42.153372 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a396fbab-7d80-43fd-acc4-8dd9dfe527ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:21:42.153894 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a396fbab-7d80-43fd-acc4-8dd9dfe527ef admin admin] PATH_INFO: `/v3/projects` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:21:42.154997 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a396fbab-7d80-43fd-acc4-8dd9dfe527ef admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:21:42.156535 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a396fbab-7d80-43fd-acc4-8dd9dfe527ef admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:21:42.301653 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1282/2587] 10.222.0.46 () {60 vars in 1153 bytes} [Mon Oct 10 12:21:42 2022] POST /identity/v3/projects => generated 363 bytes in 282 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 12:21:42.302847 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165593fc-b812-455c-aa03-5c4f49bc6839 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:21:42.314545 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165593fc-b812-455c-aa03-5c4f49bc6839 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:21:42.362401 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:21:42.397967 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9850b12a-83e2-4572-9bac-e1c47cd02d91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:21:42.398785 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9850b12a-83e2-4572-9bac-e1c47cd02d91 admin admin] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:21:42.399185 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9850b12a-83e2-4572-9bac-e1c47cd02d91 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:21:42.400032 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9850b12a-83e2-4572-9bac-e1c47cd02d91 admin admin] PATH_INFO: `/v3/users` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:21:42.400795 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9850b12a-83e2-4572-9bac-e1c47cd02d91 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:21:42.402901 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9850b12a-83e2-4572-9bac-e1c47cd02d91 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:21:42.523862 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1306/2588] 10.222.0.46 () {60 vars in 1144 bytes} [Mon Oct 10 12:21:42 2022] POST /identity/v3/users => generated 288 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 12:21:42.528080 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a396fbab-7d80-43fd-acc4-8dd9dfe527ef admin admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:21:42.542071 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a396fbab-7d80-43fd-acc4-8dd9dfe527ef admin admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:21:42.552944 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f54e46-f78c-482e-85d8-29b53a362d00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:21:42.552944 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f54e46-f78c-482e-85d8-29b53a362d00 admin admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:21:42.552944 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f54e46-f78c-482e-85d8-29b53a362d00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:21:42.552944 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f54e46-f78c-482e-85d8-29b53a362d00 admin admin] PATH_INFO: `/v3/roles` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:21:42.555210 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f54e46-f78c-482e-85d8-29b53a362d00 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:21:42.556983 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f54e46-f78c-482e-85d8-29b53a362d00 admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:21:42.564865 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9850b12a-83e2-4572-9bac-e1c47cd02d91 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:21:42.567553 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-9850b12a-83e2-4572-9bac-e1c47cd02d91 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:21:42.574752 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1283/2589] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 10 12:21:42 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 12:21:42.579719 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f54e46-f78c-482e-85d8-29b53a362d00 admin admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:21:42.583296 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9850b12a-83e2-4572-9bac-e1c47cd02d91 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:21:42.595646 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f54e46-f78c-482e-85d8-29b53a362d00 admin admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:21:42.599033 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3eb1c0-e692-4952-8aa7-fa0b30d07d27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:21:42.606217 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3eb1c0-e692-4952-8aa7-fa0b30d07d27 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:21:42.606217 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3eb1c0-e692-4952-8aa7-fa0b30d07d27 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:21:42.606217 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3eb1c0-e692-4952-8aa7-fa0b30d07d27 admin admin] PATH_INFO: `/v3/users/fef422264d0c4a43a0aa1b9eff2251bd` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:21:42.606217 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3eb1c0-e692-4952-8aa7-fa0b30d07d27 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fef422264d0c4a43a0aa1b9eff2251bd)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:21:42.606806 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62363fe-fcef-42c3-85e4-a4604a28d94c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:21:42.606806 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62363fe-fcef-42c3-85e4-a4604a28d94c admin admin] REQUEST_METHOD: `PUT` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:21:42.606806 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62363fe-fcef-42c3-85e4-a4604a28d94c admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:21:42.607052 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3eb1c0-e692-4952-8aa7-fa0b30d07d27 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:21:42.607149 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62363fe-fcef-42c3-85e4-a4604a28d94c admin admin] PATH_INFO: `/v3/projects/a460ebb268a7404994b30a0d6a2c15ea/users/4778ff9e506a4137888e5c8f13d046e6/roles/0d822a0674b6495fac623877a3b38025` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:21:42.626727 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62363fe-fcef-42c3-85e4-a4604a28d94c admin admin] RBAC: Authorizing `identity:create_grant(project_id=a460ebb268a7404994b30a0d6a2c15ea, user_id=4778ff9e506a4137888e5c8f13d046e6, role_id=0d822a0674b6495fac623877a3b38025)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:21:42.630407 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62363fe-fcef-42c3-85e4-a4604a28d94c admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:21:42.767020 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1284/2590] 10.222.0.46 () {60 vars in 1483 bytes} [Mon Oct 10 12:21:42 2022] PUT /identity/v3/projects/a460ebb268a7404994b30a0d6a2c15ea/users/4778ff9e506a4137888e5c8f13d046e6/roles/0d822a0674b6495fac623877a3b38025 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:21:42.775913 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62363fe-fcef-42c3-85e4-a4604a28d94c admin admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:21:42.793475 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=91,keystone:DELETE=4,keystone:INSERT=5 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:21:42.843675 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62363fe-fcef-42c3-85e4-a4604a28d94c admin admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:21:42.849434 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de05545-f0ee-44db-a8db-e24bc0f09c85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:21:42.850053 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de05545-f0ee-44db-a8db-e24bc0f09c85 admin admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:21:42.850345 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de05545-f0ee-44db-a8db-e24bc0f09c85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:21:42.850622 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de05545-f0ee-44db-a8db-e24bc0f09c85 admin admin] PATH_INFO: `/v3/roles` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:21:42.851313 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de05545-f0ee-44db-a8db-e24bc0f09c85 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:21:42.852957 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de05545-f0ee-44db-a8db-e24bc0f09c85 admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:21:42.864268 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1285/2591] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 10 12:21:42 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 10 12:21:42.870724 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de05545-f0ee-44db-a8db-e24bc0f09c85 admin admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:21:42.880268 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-6e3eb1c0-e692-4952-8aa7-fa0b30d07d27 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fef422264d0c4a43a0aa1b9eff2251bd'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:21:42.883443 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de05545-f0ee-44db-a8db-e24bc0f09c85 admin admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:21:42.889111 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2119b78-1743-49c2-b9dd-1192499751ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:21:42.889805 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2119b78-1743-49c2-b9dd-1192499751ef admin admin] REQUEST_METHOD: `PUT` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:21:42.890091 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2119b78-1743-49c2-b9dd-1192499751ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:21:42.890369 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2119b78-1743-49c2-b9dd-1192499751ef admin admin] PATH_INFO: `/v3/projects/a460ebb268a7404994b30a0d6a2c15ea/users/4778ff9e506a4137888e5c8f13d046e6/roles/0d822a0674b6495fac623877a3b38025` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:21:42.892666 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2119b78-1743-49c2-b9dd-1192499751ef admin admin] RBAC: Authorizing `identity:create_grant(project_id=a460ebb268a7404994b30a0d6a2c15ea, user_id=4778ff9e506a4137888e5c8f13d046e6, role_id=0d822a0674b6495fac623877a3b38025)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:21:42.894541 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-6e3eb1c0-e692-4952-8aa7-fa0b30d07d27 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fef422264d0c4a43a0aa1b9eff2251bd'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:21:42.895465 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2119b78-1743-49c2-b9dd-1192499751ef admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:21:42.896124 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-6e3eb1c0-e692-4952-8aa7-fa0b30d07d27 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fef422264d0c4a43a0aa1b9eff2251bd'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:21:42.930712 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1286/2592] 10.222.0.46 () {60 vars in 1483 bytes} [Mon Oct 10 12:21:42 2022] PUT /identity/v3/projects/a460ebb268a7404994b30a0d6a2c15ea/users/4778ff9e506a4137888e5c8f13d046e6/roles/0d822a0674b6495fac623877a3b38025 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:21:42.942457 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9407ac-d8ea-4854-a1ff-f32f6eed0f5c None None] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:21:42.943409 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9407ac-d8ea-4854-a1ff-f32f6eed0f5c None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:21:42.946334 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9407ac-d8ea-4854-a1ff-f32f6eed0f5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:21:42.996906 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.auth.core [None req-ec9407ac-d8ea-4854-a1ff-f32f6eed0f5c None None] MFA Rules not processed for user `4778ff9e506a4137888e5c8f13d046e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:21:42.998256 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-ec9407ac-d8ea-4854-a1ff-f32f6eed0f5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:21:43.010539 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-6e3eb1c0-e692-4952-8aa7-fa0b30d07d27 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fef422264d0c4a43a0aa1b9eff2251bd'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:21:43.030681 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-6e3eb1c0-e692-4952-8aa7-fa0b30d07d27 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fef422264d0c4a43a0aa1b9eff2251bd'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:21:43.044959 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1307/2593] 10.222.0.46 () {58 vars in 1224 bytes} [Mon Oct 10 12:21:42 2022] DELETE /identity/v3/users/fef422264d0c4a43a0aa1b9eff2251bd => generated 0 bytes in 479 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:21:43.112984 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1287/2594] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:21:42 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:21:43.127896 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3eb1c0-e692-4952-8aa7-fa0b30d07d27 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:21:43.130267 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-6e3eb1c0-e692-4952-8aa7-fa0b30d07d27 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:21:43.214733 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3eb1c0-e692-4952-8aa7-fa0b30d07d27 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:21:43.219575 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d604533-c336-4215-8979-0e0ea53bf018 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3ae093470484b93b23dcfae1657df9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad3772f7b6a47c28b7ba37a9316a3c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:21:43.221064 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d604533-c336-4215-8979-0e0ea53bf018 service cinder] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:21:43.221408 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d604533-c336-4215-8979-0e0ea53bf018 service cinder] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:21:43.221918 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d604533-c336-4215-8979-0e0ea53bf018 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:21:43.310113 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:21:43.313700 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9407ac-d8ea-4854-a1ff-f32f6eed0f5c None None] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:21:43.314142 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d604533-c336-4215-8979-0e0ea53bf018 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:21:43.315749 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-ec9407ac-d8ea-4854-a1ff-f32f6eed0f5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:21:43.316283 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d604533-c336-4215-8979-0e0ea53bf018 service cinder] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:21:43.330614 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1308/2595] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 10 12:21:43 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 10 12:21:43.416887 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9407ac-d8ea-4854-a1ff-f32f6eed0f5c None None] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:21:43.465325 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:21:43.508370 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f01227f-bf56-469c-8a14-1ff5dcd0cb0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:21:43.509379 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f01227f-bf56-469c-8a14-1ff5dcd0cb0a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:21:43.509764 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f01227f-bf56-469c-8a14-1ff5dcd0cb0a admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:21:43.510147 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f01227f-bf56-469c-8a14-1ff5dcd0cb0a admin admin] PATH_INFO: `/v3/projects/aab51c9b019e400c8e28d9a789eda6da` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:21:43.511568 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f01227f-bf56-469c-8a14-1ff5dcd0cb0a admin admin] RBAC: Authorizing `identity:delete_project(project_id=aab51c9b019e400c8e28d9a789eda6da)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:21:43.513873 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f01227f-bf56-469c-8a14-1ff5dcd0cb0a admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:21:43.570276 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d604533-c336-4215-8979-0e0ea53bf018 service cinder] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:21:43.581227 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d604533-c336-4215-8979-0e0ea53bf018 service cinder] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:21:43.585428 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208c9f9f-724a-4fd0-b8f7-ed41ff6dd59e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3ae093470484b93b23dcfae1657df9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad3772f7b6a47c28b7ba37a9316a3c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:21:43.586760 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208c9f9f-724a-4fd0-b8f7-ed41ff6dd59e service cinder] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:21:43.587115 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208c9f9f-724a-4fd0-b8f7-ed41ff6dd59e service cinder] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:21:43.587478 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208c9f9f-724a-4fd0-b8f7-ed41ff6dd59e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:21:43.589384 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-208c9f9f-724a-4fd0-b8f7-ed41ff6dd59e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:21:43.670060 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208c9f9f-724a-4fd0-b8f7-ed41ff6dd59e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:21:43.672133 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208c9f9f-724a-4fd0-b8f7-ed41ff6dd59e service cinder] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:21:43.675247 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-4f01227f-bf56-469c-8a14-1ff5dcd0cb0a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aab51c9b019e400c8e28d9a789eda6da', 'request_id': 'req-4f01227f-bf56-469c-8a14-1ff5dcd0cb0a'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:21:43.685005 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-4f01227f-bf56-469c-8a14-1ff5dcd0cb0a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aab51c9b019e400c8e28d9a789eda6da', 'request_id': 'req-4f01227f-bf56-469c-8a14-1ff5dcd0cb0a'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:21:43.692452 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-4f01227f-bf56-469c-8a14-1ff5dcd0cb0a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aab51c9b019e400c8e28d9a789eda6da', 'request_id': 'req-4f01227f-bf56-469c-8a14-1ff5dcd0cb0a'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:21:43.698766 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1309/2596] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 10 12:21:43 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 10 12:21:43.761864 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-4f01227f-bf56-469c-8a14-1ff5dcd0cb0a admin admin] The token cache is being invalidate because project aab51c9b019e400c8e28d9a789eda6da was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81676) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 10 12:21:43.762160 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-4f01227f-bf56-469c-8a14-1ff5dcd0cb0a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:21:43.767794 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-4f01227f-bf56-469c-8a14-1ff5dcd0cb0a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:21:43.767794 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1288/2597] 10.222.0.46 () {58 vars in 1233 bytes} [Mon Oct 10 12:21:43 2022] DELETE /identity/v3/projects/aab51c9b019e400c8e28d9a789eda6da => generated 0 bytes in 452 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:21:43.770137 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b67e86-f76b-41c0-b852-647041071a25 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:21:43.770426 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b67e86-f76b-41c0-b852-647041071a25 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:21:43.770718 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b67e86-f76b-41c0-b852-647041071a25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:21:43.808650 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-34b67e86-f76b-41c0-b852-647041071a25 None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:21:43.813117 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-34b67e86-f76b-41c0-b852-647041071a25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:21:43.879543 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1310/2598] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:21:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:21:43.889791 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186c2b8a-ee8d-4daf-9a25-99b43124abbb None None] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:21:43.893579 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186c2b8a-ee8d-4daf-9a25-99b43124abbb None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:21:43.893579 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186c2b8a-ee8d-4daf-9a25-99b43124abbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:21:43.911929 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.auth.core [None req-186c2b8a-ee8d-4daf-9a25-99b43124abbb None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:21:43.913079 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-186c2b8a-ee8d-4daf-9a25-99b43124abbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:21:43.922360 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1289/2599] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:21:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:21:43.927937 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b67e86-f76b-41c0-b852-647041071a25 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:21:43.942007 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b67e86-f76b-41c0-b852-647041071a25 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:21:43.947607 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842b8c7f-b3ed-42db-b167-6d889e176e2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:21:43.948334 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842b8c7f-b3ed-42db-b167-6d889e176e2e admin admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:21:43.948723 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842b8c7f-b3ed-42db-b167-6d889e176e2e admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:21:43.949069 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842b8c7f-b3ed-42db-b167-6d889e176e2e admin admin] PATH_INFO: `/v3/domains` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:21:43.949760 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842b8c7f-b3ed-42db-b167-6d889e176e2e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 12:21:43.950109 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842b8c7f-b3ed-42db-b167-6d889e176e2e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:21:43.951863 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842b8c7f-b3ed-42db-b167-6d889e176e2e admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:21:43.966803 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1311/2600] 10.222.0.46 () {58 vars in 1153 bytes} [Mon Oct 10 12:21:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 12:21:44.002109 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186c2b8a-ee8d-4daf-9a25-99b43124abbb None None] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:21:44.011603 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186c2b8a-ee8d-4daf-9a25-99b43124abbb None None] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:21:44.016629 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9364cb02-d7f6-4786-bfee-93a3cb9b67b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:21:44.017260 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9364cb02-d7f6-4786-bfee-93a3cb9b67b0 admin admin] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:21:44.017543 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9364cb02-d7f6-4786-bfee-93a3cb9b67b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:21:44.017812 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9364cb02-d7f6-4786-bfee-93a3cb9b67b0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:21:44.018449 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9364cb02-d7f6-4786-bfee-93a3cb9b67b0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:21:44.020478 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9364cb02-d7f6-4786-bfee-93a3cb9b67b0 admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:21:44.215736 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1290/2601] 10.222.0.46 () {60 vars in 1153 bytes} [Mon Oct 10 12:21:43 2022] POST /identity/v3/projects => generated 385 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 12:21:44.217175 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842b8c7f-b3ed-42db-b167-6d889e176e2e admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:21:44.228829 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842b8c7f-b3ed-42db-b167-6d889e176e2e admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:21:44.319480 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e7af02-a370-470f-9473-9e9c34b52d54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:21:44.320216 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e7af02-a370-470f-9473-9e9c34b52d54 admin admin] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:21:44.320759 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e7af02-a370-470f-9473-9e9c34b52d54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:21:44.321023 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e7af02-a370-470f-9473-9e9c34b52d54 admin admin] PATH_INFO: `/v3/users` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:21:44.322143 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e7af02-a370-470f-9473-9e9c34b52d54 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:21:44.324401 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e7af02-a370-470f-9473-9e9c34b52d54 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:21:44.414904 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1312/2602] 10.222.0.46 () {60 vars in 1144 bytes} [Mon Oct 10 12:21:44 2022] POST /identity/v3/users => generated 293 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 12:21:44.421271 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9364cb02-d7f6-4786-bfee-93a3cb9b67b0 admin admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:21:44.434072 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9364cb02-d7f6-4786-bfee-93a3cb9b67b0 admin admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:21:44.443574 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0139c518-b129-4349-8c9e-de13c13eb8e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:21:44.444347 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0139c518-b129-4349-8c9e-de13c13eb8e7 admin admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:21:44.444670 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0139c518-b129-4349-8c9e-de13c13eb8e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:21:44.444953 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0139c518-b129-4349-8c9e-de13c13eb8e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:21:44.445620 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0139c518-b129-4349-8c9e-de13c13eb8e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:21:44.447277 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0139c518-b129-4349-8c9e-de13c13eb8e7 admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:21:44.462166 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1291/2603] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 10 12:21:44 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 12:21:44.463494 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e7af02-a370-470f-9473-9e9c34b52d54 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:21:44.477727 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e7af02-a370-470f-9473-9e9c34b52d54 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:21:44.482636 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c00b503-d6c9-41ff-ae8f-ad8f9cb9bab4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:21:44.483277 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c00b503-d6c9-41ff-ae8f-ad8f9cb9bab4 admin admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:21:44.486972 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c00b503-d6c9-41ff-ae8f-ad8f9cb9bab4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:21:44.487329 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c00b503-d6c9-41ff-ae8f-ad8f9cb9bab4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:21:44.488036 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c00b503-d6c9-41ff-ae8f-ad8f9cb9bab4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:21:44.489865 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c00b503-d6c9-41ff-ae8f-ad8f9cb9bab4 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:21:44.496429 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:21:44.502151 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1313/2604] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 10 12:21:44 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 12:21:44.506358 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0139c518-b129-4349-8c9e-de13c13eb8e7 admin admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:21:44.516704 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0139c518-b129-4349-8c9e-de13c13eb8e7 admin admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:21:44.525463 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6655480-7fc9-4ecf-a884-5726803914aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:21:44.530842 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6655480-7fc9-4ecf-a884-5726803914aa admin admin] REQUEST_METHOD: `PUT` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:21:44.530842 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6655480-7fc9-4ecf-a884-5726803914aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:21:44.530842 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6655480-7fc9-4ecf-a884-5726803914aa admin admin] PATH_INFO: `/v3/projects/4bd4f7bcd357441185ee61871404c84b/users/ab50583207134c8fa2026cc245d252c2/roles/b2cb2a6e818e47338ce071e6fedfc673` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:21:44.543027 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6655480-7fc9-4ecf-a884-5726803914aa admin admin] RBAC: Authorizing `identity:create_grant(project_id=4bd4f7bcd357441185ee61871404c84b, user_id=ab50583207134c8fa2026cc245d252c2, role_id=b2cb2a6e818e47338ce071e6fedfc673)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:21:44.545724 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6655480-7fc9-4ecf-a884-5726803914aa admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:21:44.635393 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1292/2605] 10.222.0.46 () {60 vars in 1483 bytes} [Mon Oct 10 12:21:44 2022] PUT /identity/v3/projects/4bd4f7bcd357441185ee61871404c84b/users/ab50583207134c8fa2026cc245d252c2/roles/b2cb2a6e818e47338ce071e6fedfc673 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:21:44.646443 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddcf1f5-2183-49ac-9ec7-d89d0b49e4d4 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:21:44.647213 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddcf1f5-2183-49ac-9ec7-d89d0b49e4d4 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:21:44.647213 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddcf1f5-2183-49ac-9ec7-d89d0b49e4d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:21:44.693476 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-dddcf1f5-2183-49ac-9ec7-d89d0b49e4d4 None None] MFA Rules not processed for user `ab50583207134c8fa2026cc245d252c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:21:44.694903 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-dddcf1f5-2183-49ac-9ec7-d89d0b49e4d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:21:44.780751 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1314/2606] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:21:44 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 10 12:21:44.794857 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6655480-7fc9-4ecf-a884-5726803914aa admin admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:21:44.796705 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-d6655480-7fc9-4ecf-a884-5726803914aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:21:44.865988 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:21:44.885622 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6655480-7fc9-4ecf-a884-5726803914aa admin admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:21:44.891485 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf38bd3-e533-4248-a1e0-a3a79b3beb64 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f130bbb69e546ce99b4e14de81d7e54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad3772f7b6a47c28b7ba37a9316a3c1', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:21:44.892168 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf38bd3-e533-4248-a1e0-a3a79b3beb64 service nova] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:21:44.892385 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf38bd3-e533-4248-a1e0-a3a79b3beb64 service nova] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:21:44.892662 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf38bd3-e533-4248-a1e0-a3a79b3beb64 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:21:44.903427 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf38bd3-e533-4248-a1e0-a3a79b3beb64 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:21:44.903427 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf38bd3-e533-4248-a1e0-a3a79b3beb64 service nova] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:21:44.914308 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1293/2607] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 10 12:21:44 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 10 12:21:45.743760 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddcf1f5-2183-49ac-9ec7-d89d0b49e4d4 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:21:45.745219 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-dddcf1f5-2183-49ac-9ec7-d89d0b49e4d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:21:45.844450 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddcf1f5-2183-49ac-9ec7-d89d0b49e4d4 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:21:45.850204 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b6c1fb-b7bf-4e4c-bfc4-7b68a62bfd4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:21:45.850789 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b6c1fb-b7bf-4e4c-bfc4-7b68a62bfd4d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:21:45.854857 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b6c1fb-b7bf-4e4c-bfc4-7b68a62bfd4d admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:21:45.854857 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b6c1fb-b7bf-4e4c-bfc4-7b68a62bfd4d admin admin] PATH_INFO: `/v3/users/21c537b0fefe4fb1b51483ac0e3cf2d1` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:21:45.859343 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b6c1fb-b7bf-4e4c-bfc4-7b68a62bfd4d admin admin] RBAC: Authorizing `identity:delete_user(user_id=21c537b0fefe4fb1b51483ac0e3cf2d1)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:21:45.859343 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b6c1fb-b7bf-4e4c-bfc4-7b68a62bfd4d admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:21:46.147216 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-11b6c1fb-b7bf-4e4c-bfc4-7b68a62bfd4d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '21c537b0fefe4fb1b51483ac0e3cf2d1'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:21:46.155428 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-11b6c1fb-b7bf-4e4c-bfc4-7b68a62bfd4d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '21c537b0fefe4fb1b51483ac0e3cf2d1'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:21:46.156140 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-11b6c1fb-b7bf-4e4c-bfc4-7b68a62bfd4d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '21c537b0fefe4fb1b51483ac0e3cf2d1'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:21:46.233326 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-11b6c1fb-b7bf-4e4c-bfc4-7b68a62bfd4d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '21c537b0fefe4fb1b51483ac0e3cf2d1'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:21:46.247138 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-11b6c1fb-b7bf-4e4c-bfc4-7b68a62bfd4d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '21c537b0fefe4fb1b51483ac0e3cf2d1'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:21:46.254096 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1315/2608] 10.222.0.46 () {58 vars in 1224 bytes} [Mon Oct 10 12:21:45 2022] DELETE /identity/v3/users/21c537b0fefe4fb1b51483ac0e3cf2d1 => generated 0 bytes in 511 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:21:46.544248 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf38bd3-e533-4248-a1e0-a3a79b3beb64 service nova] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:21:46.559704 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf38bd3-e533-4248-a1e0-a3a79b3beb64 service nova] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:21:46.628433 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988f8689-9e65-4569-b688-50b14e7a8187 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:21:46.629075 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988f8689-9e65-4569-b688-50b14e7a8187 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:21:46.629361 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988f8689-9e65-4569-b688-50b14e7a8187 admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:21:46.629639 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988f8689-9e65-4569-b688-50b14e7a8187 admin admin] PATH_INFO: `/v3/projects/26773079b6914a20ab27109b0dec496a` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:21:46.630830 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988f8689-9e65-4569-b688-50b14e7a8187 admin admin] RBAC: Authorizing `identity:delete_project(project_id=26773079b6914a20ab27109b0dec496a)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:21:46.632809 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988f8689-9e65-4569-b688-50b14e7a8187 admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:21:46.799852 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-988f8689-9e65-4569-b688-50b14e7a8187 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '26773079b6914a20ab27109b0dec496a', 'request_id': 'req-988f8689-9e65-4569-b688-50b14e7a8187'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:21:46.806113 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-988f8689-9e65-4569-b688-50b14e7a8187 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '26773079b6914a20ab27109b0dec496a', 'request_id': 'req-988f8689-9e65-4569-b688-50b14e7a8187'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:21:46.813167 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-988f8689-9e65-4569-b688-50b14e7a8187 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '26773079b6914a20ab27109b0dec496a', 'request_id': 'req-988f8689-9e65-4569-b688-50b14e7a8187'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:21:46.940225 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-988f8689-9e65-4569-b688-50b14e7a8187 admin admin] The token cache is being invalidate because project 26773079b6914a20ab27109b0dec496a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81676) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 10 12:21:46.940225 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-988f8689-9e65-4569-b688-50b14e7a8187 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:21:46.940225 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-988f8689-9e65-4569-b688-50b14e7a8187 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:21:46.940225 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1294/2609] 10.222.0.46 () {58 vars in 1233 bytes} [Mon Oct 10 12:21:46 2022] DELETE /identity/v3/projects/26773079b6914a20ab27109b0dec496a => generated 0 bytes in 397 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:21:46.944902 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b6c1fb-b7bf-4e4c-bfc4-7b68a62bfd4d admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:21:46.947032 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-11b6c1fb-b7bf-4e4c-bfc4-7b68a62bfd4d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:21:46.964994 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b6c1fb-b7bf-4e4c-bfc4-7b68a62bfd4d admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:21:46.972922 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:21:47.035981 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d2e36c-aa06-4f1f-8760-53f795406399 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:21:47.036935 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d2e36c-aa06-4f1f-8760-53f795406399 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:21:47.037425 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d2e36c-aa06-4f1f-8760-53f795406399 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:21:47.037717 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d2e36c-aa06-4f1f-8760-53f795406399 admin admin] PATH_INFO: `/v3/users/09668960625845f0ad79bb91a6de7766` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:21:47.053069 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d2e36c-aa06-4f1f-8760-53f795406399 admin admin] RBAC: Authorizing `identity:delete_user(user_id=09668960625845f0ad79bb91a6de7766)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:21:47.055526 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d2e36c-aa06-4f1f-8760-53f795406399 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:21:47.342461 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-46d2e36c-aa06-4f1f-8760-53f795406399 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '09668960625845f0ad79bb91a6de7766'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:21:47.350681 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-46d2e36c-aa06-4f1f-8760-53f795406399 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '09668960625845f0ad79bb91a6de7766'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:21:47.351411 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-46d2e36c-aa06-4f1f-8760-53f795406399 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '09668960625845f0ad79bb91a6de7766'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:21:47.447385 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-46d2e36c-aa06-4f1f-8760-53f795406399 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '09668960625845f0ad79bb91a6de7766'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:21:47.461415 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-46d2e36c-aa06-4f1f-8760-53f795406399 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '09668960625845f0ad79bb91a6de7766'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:21:47.470802 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1316/2610] 10.222.0.46 () {58 vars in 1224 bytes} [Mon Oct 10 12:21:46 2022] DELETE /identity/v3/users/09668960625845f0ad79bb91a6de7766 => generated 0 bytes in 526 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 10 12:21:47.802636 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988f8689-9e65-4569-b688-50b14e7a8187 admin admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:21:47.805292 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-988f8689-9e65-4569-b688-50b14e7a8187 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:21:47.819399 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988f8689-9e65-4569-b688-50b14e7a8187 admin admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:21:47.882009 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ddea4ee-038f-4974-9207-0a8d5063e676 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:21:47.882837 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddea4ee-038f-4974-9207-0a8d5063e676 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:21:47.883123 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddea4ee-038f-4974-9207-0a8d5063e676 admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:21:47.883402 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddea4ee-038f-4974-9207-0a8d5063e676 admin admin] PATH_INFO: `/v3/projects/568ccc02e08d4bc6bccfe42c58fcf97e` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:21:47.894885 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ddea4ee-038f-4974-9207-0a8d5063e676 admin admin] RBAC: Authorizing `identity:delete_project(project_id=568ccc02e08d4bc6bccfe42c58fcf97e)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:21:47.896837 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ddea4ee-038f-4974-9207-0a8d5063e676 admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:21:47.899370 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:21:47.925897 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d2e36c-aa06-4f1f-8760-53f795406399 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:21:47.927406 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-46d2e36c-aa06-4f1f-8760-53f795406399 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:21:47.940504 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d2e36c-aa06-4f1f-8760-53f795406399 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:21:47.944987 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9cdd46b-f7e7-4093-917d-3aa764256416 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:21:47.945605 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cdd46b-f7e7-4093-917d-3aa764256416 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:21:47.945881 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cdd46b-f7e7-4093-917d-3aa764256416 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:21:47.946154 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cdd46b-f7e7-4093-917d-3aa764256416 admin admin] PATH_INFO: `/v3/users/5185828d5a1149369fce35efd4c3e8b4` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:21:47.947311 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9cdd46b-f7e7-4093-917d-3aa764256416 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5185828d5a1149369fce35efd4c3e8b4)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:21:47.949192 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9cdd46b-f7e7-4093-917d-3aa764256416 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:21:48.013960 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-6ddea4ee-038f-4974-9207-0a8d5063e676 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '568ccc02e08d4bc6bccfe42c58fcf97e', 'request_id': 'req-6ddea4ee-038f-4974-9207-0a8d5063e676'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:21:48.020269 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-6ddea4ee-038f-4974-9207-0a8d5063e676 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '568ccc02e08d4bc6bccfe42c58fcf97e', 'request_id': 'req-6ddea4ee-038f-4974-9207-0a8d5063e676'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:21:48.027087 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-6ddea4ee-038f-4974-9207-0a8d5063e676 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '568ccc02e08d4bc6bccfe42c58fcf97e', 'request_id': 'req-6ddea4ee-038f-4974-9207-0a8d5063e676'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:21:48.116262 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-6ddea4ee-038f-4974-9207-0a8d5063e676 admin admin] The token cache is being invalidate because project 568ccc02e08d4bc6bccfe42c58fcf97e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81676) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 10 12:21:48.116262 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-6ddea4ee-038f-4974-9207-0a8d5063e676 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:21:48.116262 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-6ddea4ee-038f-4974-9207-0a8d5063e676 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:21:48.116262 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1295/2611] 10.222.0.46 () {58 vars in 1233 bytes} [Mon Oct 10 12:21:47 2022] DELETE /identity/v3/projects/568ccc02e08d4bc6bccfe42c58fcf97e => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:21:48.177500 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-b9cdd46b-f7e7-4093-917d-3aa764256416 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5185828d5a1149369fce35efd4c3e8b4'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:21:48.185431 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-b9cdd46b-f7e7-4093-917d-3aa764256416 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5185828d5a1149369fce35efd4c3e8b4'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:21:48.186291 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-b9cdd46b-f7e7-4093-917d-3aa764256416 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5185828d5a1149369fce35efd4c3e8b4'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:21:48.276825 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-b9cdd46b-f7e7-4093-917d-3aa764256416 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5185828d5a1149369fce35efd4c3e8b4'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:21:48.285834 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:21:48.292302 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-b9cdd46b-f7e7-4093-917d-3aa764256416 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5185828d5a1149369fce35efd4c3e8b4'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:21:48.300151 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1317/2612] 10.222.0.46 () {58 vars in 1224 bytes} [Mon Oct 10 12:21:47 2022] DELETE /identity/v3/users/5185828d5a1149369fce35efd4c3e8b4 => generated 0 bytes in 375 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 10 12:21:48.305350 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0825202f-fda5-4ec6-881a-8427d6c91dd8 None None] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:21:48.305653 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0825202f-fda5-4ec6-881a-8427d6c91dd8 None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:21:48.306610 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0825202f-fda5-4ec6-881a-8427d6c91dd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:21:48.328801 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.auth.core [None req-0825202f-fda5-4ec6-881a-8427d6c91dd8 None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:21:48.330180 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-0825202f-fda5-4ec6-881a-8427d6c91dd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:21:48.401520 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1296/2613] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:21:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:21:48.411199 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9cdd46b-f7e7-4093-917d-3aa764256416 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:21:48.413383 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-b9cdd46b-f7e7-4093-917d-3aa764256416 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:21:48.472705 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9cdd46b-f7e7-4093-917d-3aa764256416 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:21:48.476208 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872ace89-de40-4baa-869a-aaa761b9c40f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58b1c7c85302421cba823ace4201861f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad3772f7b6a47c28b7ba37a9316a3c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:21:48.477026 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872ace89-de40-4baa-869a-aaa761b9c40f service neutron] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:21:48.477315 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872ace89-de40-4baa-869a-aaa761b9c40f service neutron] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:21:48.477591 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872ace89-de40-4baa-869a-aaa761b9c40f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:21:48.487892 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872ace89-de40-4baa-869a-aaa761b9c40f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:21:48.493744 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872ace89-de40-4baa-869a-aaa761b9c40f service neutron] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:21:48.498993 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1318/2614] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 10 12:21:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:21:48.930086 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0825202f-fda5-4ec6-881a-8427d6c91dd8 None None] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:21:48.931710 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-0825202f-fda5-4ec6-881a-8427d6c91dd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:21:48.947236 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0825202f-fda5-4ec6-881a-8427d6c91dd8 None None] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:21:48.953189 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372b953d-1f3d-4ed5-aa37-6430dd3c36df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:21:48.953893 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372b953d-1f3d-4ed5-aa37-6430dd3c36df admin admin] REQUEST_METHOD: `DELETE` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:21:48.954067 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372b953d-1f3d-4ed5-aa37-6430dd3c36df admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:21:48.954340 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372b953d-1f3d-4ed5-aa37-6430dd3c36df admin admin] PATH_INFO: `/v3/projects/ac4567ab040f46929f2d39006b0a9d5a` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:21:48.955623 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372b953d-1f3d-4ed5-aa37-6430dd3c36df admin admin] RBAC: Authorizing `identity:delete_project(project_id=ac4567ab040f46929f2d39006b0a9d5a)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:21:48.957510 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372b953d-1f3d-4ed5-aa37-6430dd3c36df admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:21:49.076841 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-372b953d-1f3d-4ed5-aa37-6430dd3c36df admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ac4567ab040f46929f2d39006b0a9d5a', 'request_id': 'req-372b953d-1f3d-4ed5-aa37-6430dd3c36df'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:21:49.083053 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-372b953d-1f3d-4ed5-aa37-6430dd3c36df admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ac4567ab040f46929f2d39006b0a9d5a', 'request_id': 'req-372b953d-1f3d-4ed5-aa37-6430dd3c36df'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:21:49.089619 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-372b953d-1f3d-4ed5-aa37-6430dd3c36df admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ac4567ab040f46929f2d39006b0a9d5a', 'request_id': 'req-372b953d-1f3d-4ed5-aa37-6430dd3c36df'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:21:49.164229 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-372b953d-1f3d-4ed5-aa37-6430dd3c36df admin admin] The token cache is being invalidate because project ac4567ab040f46929f2d39006b0a9d5a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81676) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 10 12:21:49.164512 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-372b953d-1f3d-4ed5-aa37-6430dd3c36df admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:21:49.165126 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-372b953d-1f3d-4ed5-aa37-6430dd3c36df admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:21:49.166613 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1297/2615] 10.222.0.46 () {58 vars in 1233 bytes} [Mon Oct 10 12:21:48 2022] DELETE /identity/v3/projects/ac4567ab040f46929f2d39006b0a9d5a => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:21:49.170620 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872ace89-de40-4baa-869a-aaa761b9c40f service neutron] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:21:49.180419 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872ace89-de40-4baa-869a-aaa761b9c40f service neutron] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:21:49.236512 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34113a8f-fa7d-44ef-9d9d-5f6d20ea6e29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:21:49.237057 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34113a8f-fa7d-44ef-9d9d-5f6d20ea6e29 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:21:49.237334 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34113a8f-fa7d-44ef-9d9d-5f6d20ea6e29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:21:49.237605 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34113a8f-fa7d-44ef-9d9d-5f6d20ea6e29 admin admin] PATH_INFO: `/v3/users/4778ff9e506a4137888e5c8f13d046e6` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:21:49.238730 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34113a8f-fa7d-44ef-9d9d-5f6d20ea6e29 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4778ff9e506a4137888e5c8f13d046e6)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:21:49.240919 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34113a8f-fa7d-44ef-9d9d-5f6d20ea6e29 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:21:49.419075 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-34113a8f-fa7d-44ef-9d9d-5f6d20ea6e29 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4778ff9e506a4137888e5c8f13d046e6'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:21:49.426155 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-34113a8f-fa7d-44ef-9d9d-5f6d20ea6e29 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4778ff9e506a4137888e5c8f13d046e6'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:21:49.426877 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-34113a8f-fa7d-44ef-9d9d-5f6d20ea6e29 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4778ff9e506a4137888e5c8f13d046e6'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:21:49.504896 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-34113a8f-fa7d-44ef-9d9d-5f6d20ea6e29 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4778ff9e506a4137888e5c8f13d046e6'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:21:49.518373 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-34113a8f-fa7d-44ef-9d9d-5f6d20ea6e29 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4778ff9e506a4137888e5c8f13d046e6'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:21:49.524318 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1319/2616] 10.222.0.46 () {58 vars in 1224 bytes} [Mon Oct 10 12:21:49 2022] DELETE /identity/v3/users/4778ff9e506a4137888e5c8f13d046e6 => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:21:49.924288 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372b953d-1f3d-4ed5-aa37-6430dd3c36df admin admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:21:49.925703 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-372b953d-1f3d-4ed5-aa37-6430dd3c36df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:21:49.939977 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372b953d-1f3d-4ed5-aa37-6430dd3c36df admin admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:21:49.952995 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:21:50.004364 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78618d55-ec45-4bd6-b935-e2edfde7f3db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:21:50.006048 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78618d55-ec45-4bd6-b935-e2edfde7f3db admin admin] REQUEST_METHOD: `DELETE` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:21:50.006048 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78618d55-ec45-4bd6-b935-e2edfde7f3db admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:21:50.006048 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78618d55-ec45-4bd6-b935-e2edfde7f3db admin admin] PATH_INFO: `/v3/projects/a460ebb268a7404994b30a0d6a2c15ea` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:21:50.007188 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78618d55-ec45-4bd6-b935-e2edfde7f3db admin admin] RBAC: Authorizing `identity:delete_project(project_id=a460ebb268a7404994b30a0d6a2c15ea)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:21:50.009284 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78618d55-ec45-4bd6-b935-e2edfde7f3db admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:21:50.149448 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-78618d55-ec45-4bd6-b935-e2edfde7f3db admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a460ebb268a7404994b30a0d6a2c15ea', 'request_id': 'req-78618d55-ec45-4bd6-b935-e2edfde7f3db'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:21:50.155532 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-78618d55-ec45-4bd6-b935-e2edfde7f3db admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a460ebb268a7404994b30a0d6a2c15ea', 'request_id': 'req-78618d55-ec45-4bd6-b935-e2edfde7f3db'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:21:50.162927 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-78618d55-ec45-4bd6-b935-e2edfde7f3db admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a460ebb268a7404994b30a0d6a2c15ea', 'request_id': 'req-78618d55-ec45-4bd6-b935-e2edfde7f3db'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:21:50.244897 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-78618d55-ec45-4bd6-b935-e2edfde7f3db admin admin] The token cache is being invalidate because project a460ebb268a7404994b30a0d6a2c15ea was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81676) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 10 12:21:50.245191 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-78618d55-ec45-4bd6-b935-e2edfde7f3db admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:21:50.245856 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-78618d55-ec45-4bd6-b935-e2edfde7f3db admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:21:50.247480 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1298/2617] 10.222.0.46 () {58 vars in 1233 bytes} [Mon Oct 10 12:21:49 2022] DELETE /identity/v3/projects/a460ebb268a7404994b30a0d6a2c15ea => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:21:50.252696 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042f281c-355e-4288-9de7-b252cdfc0ea1 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:21:50.253481 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042f281c-355e-4288-9de7-b252cdfc0ea1 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:21:50.253481 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042f281c-355e-4288-9de7-b252cdfc0ea1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:21:50.274452 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-042f281c-355e-4288-9de7-b252cdfc0ea1 None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:21:50.275704 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-042f281c-355e-4288-9de7-b252cdfc0ea1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:21:50.281227 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:21:50.338045 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1320/2618] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:21:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:21:50.347329 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b884ed3-e37c-4aaf-b001-ca6c58f28de2 None None] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:21:50.347643 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b884ed3-e37c-4aaf-b001-ca6c58f28de2 None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:21:50.351203 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b884ed3-e37c-4aaf-b001-ca6c58f28de2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:21:50.368605 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.auth.core [None req-4b884ed3-e37c-4aaf-b001-ca6c58f28de2 None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:21:50.369807 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-4b884ed3-e37c-4aaf-b001-ca6c58f28de2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:21:50.377943 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1299/2619] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:21:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:21:50.382610 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042f281c-355e-4288-9de7-b252cdfc0ea1 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:21:50.392398 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042f281c-355e-4288-9de7-b252cdfc0ea1 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:21:50.397434 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa31263-df35-4d50-8ec1-383bd87843ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:21:50.398039 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa31263-df35-4d50-8ec1-383bd87843ca admin admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:21:50.398323 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa31263-df35-4d50-8ec1-383bd87843ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:21:50.398601 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa31263-df35-4d50-8ec1-383bd87843ca admin admin] PATH_INFO: `/v3/domains` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:21:50.399281 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa31263-df35-4d50-8ec1-383bd87843ca admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 12:21:50.399657 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa31263-df35-4d50-8ec1-383bd87843ca admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:21:50.401285 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa31263-df35-4d50-8ec1-383bd87843ca admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:21:50.414370 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1321/2620] 10.222.0.46 () {58 vars in 1153 bytes} [Mon Oct 10 12:21:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 12:21:50.419222 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b884ed3-e37c-4aaf-b001-ca6c58f28de2 None None] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:21:50.429203 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b884ed3-e37c-4aaf-b001-ca6c58f28de2 None None] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:21:50.434175 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efc89bd-97cd-4b50-b243-a3f0d3060d7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:21:50.435526 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efc89bd-97cd-4b50-b243-a3f0d3060d7e admin admin] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:21:50.435809 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efc89bd-97cd-4b50-b243-a3f0d3060d7e admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:21:50.436095 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efc89bd-97cd-4b50-b243-a3f0d3060d7e admin admin] PATH_INFO: `/v3/projects` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:21:50.436752 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2efc89bd-97cd-4b50-b243-a3f0d3060d7e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:21:50.438734 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2efc89bd-97cd-4b50-b243-a3f0d3060d7e admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:21:50.537709 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1300/2621] 10.222.0.46 () {60 vars in 1153 bytes} [Mon Oct 10 12:21:50 2022] POST /identity/v3/projects => generated 385 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 12:21:50.541953 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa31263-df35-4d50-8ec1-383bd87843ca admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:21:50.553794 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa31263-df35-4d50-8ec1-383bd87843ca admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:21:50.612366 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81003007-d918-4882-b177-06ed149b910c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:21:50.612978 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81003007-d918-4882-b177-06ed149b910c admin admin] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:21:50.613257 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81003007-d918-4882-b177-06ed149b910c admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:21:50.613643 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81003007-d918-4882-b177-06ed149b910c admin admin] PATH_INFO: `/v3/users` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:21:50.614282 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81003007-d918-4882-b177-06ed149b910c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:21:50.616229 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81003007-d918-4882-b177-06ed149b910c admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:21:50.697001 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1322/2622] 10.222.0.46 () {60 vars in 1144 bytes} [Mon Oct 10 12:21:50 2022] POST /identity/v3/users => generated 293 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 12:21:50.701040 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efc89bd-97cd-4b50-b243-a3f0d3060d7e admin admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:21:50.711946 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efc89bd-97cd-4b50-b243-a3f0d3060d7e admin admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:21:50.716669 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a82097-c949-41d1-bbab-7bccca293db9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:21:50.717266 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a82097-c949-41d1-bbab-7bccca293db9 admin admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:21:50.717552 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a82097-c949-41d1-bbab-7bccca293db9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:21:50.717826 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a82097-c949-41d1-bbab-7bccca293db9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:21:50.718473 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a82097-c949-41d1-bbab-7bccca293db9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:21:50.720218 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a82097-c949-41d1-bbab-7bccca293db9 admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:21:50.730490 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1301/2623] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 10 12:21:50 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 10 12:21:50.734482 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81003007-d918-4882-b177-06ed149b910c admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:21:50.745949 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81003007-d918-4882-b177-06ed149b910c admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:21:50.750551 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6baf8543-f8b1-4419-9dbd-caee3fe89139 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:21:50.751788 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6baf8543-f8b1-4419-9dbd-caee3fe89139 admin admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:21:50.752067 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6baf8543-f8b1-4419-9dbd-caee3fe89139 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:21:50.752341 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6baf8543-f8b1-4419-9dbd-caee3fe89139 admin admin] PATH_INFO: `/v3/roles` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:21:50.752986 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6baf8543-f8b1-4419-9dbd-caee3fe89139 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:21:50.754561 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6baf8543-f8b1-4419-9dbd-caee3fe89139 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:21:50.764528 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1323/2624] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 10 12:21:50 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 12:21:50.768661 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a82097-c949-41d1-bbab-7bccca293db9 admin admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:21:50.778694 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a82097-c949-41d1-bbab-7bccca293db9 admin admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:21:50.783270 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e6699b-03ea-4d2b-aaa9-5f94bc5d8aac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:21:50.783917 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e6699b-03ea-4d2b-aaa9-5f94bc5d8aac admin admin] REQUEST_METHOD: `PUT` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:21:50.784202 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e6699b-03ea-4d2b-aaa9-5f94bc5d8aac admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:21:50.784481 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e6699b-03ea-4d2b-aaa9-5f94bc5d8aac admin admin] PATH_INFO: `/v3/projects/db0440d186134397809e219a27deff13/users/af6a0f97f1a2481ea3a573ecb2560e90/roles/b2cb2a6e818e47338ce071e6fedfc673` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:21:50.799222 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e6699b-03ea-4d2b-aaa9-5f94bc5d8aac admin admin] RBAC: Authorizing `identity:create_grant(project_id=db0440d186134397809e219a27deff13, user_id=af6a0f97f1a2481ea3a573ecb2560e90, role_id=b2cb2a6e818e47338ce071e6fedfc673)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:21:50.801879 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e6699b-03ea-4d2b-aaa9-5f94bc5d8aac admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:21:50.922288 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1302/2625] 10.222.0.46 () {60 vars in 1483 bytes} [Mon Oct 10 12:21:50 2022] PUT /identity/v3/projects/db0440d186134397809e219a27deff13/users/af6a0f97f1a2481ea3a573ecb2560e90/roles/b2cb2a6e818e47338ce071e6fedfc673 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 10 12:21:50.934502 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7ebbaf-97e2-4467-b772-7ab939ce889f None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:21:50.934822 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7ebbaf-97e2-4467-b772-7ab939ce889f None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:21:50.935096 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7ebbaf-97e2-4467-b772-7ab939ce889f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:21:50.963910 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-4e7ebbaf-97e2-4467-b772-7ab939ce889f None None] MFA Rules not processed for user `af6a0f97f1a2481ea3a573ecb2560e90`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:21:50.965036 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-4e7ebbaf-97e2-4467-b772-7ab939ce889f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:21:51.029863 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1324/2626] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:21:50 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:21:51.035095 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e6699b-03ea-4d2b-aaa9-5f94bc5d8aac admin admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:21:51.080994 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:21:51.098601 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e6699b-03ea-4d2b-aaa9-5f94bc5d8aac admin admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:21:51.103898 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b347351-93f7-4a96-9f42-0e82c0a3c4c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:21:51.105033 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b347351-93f7-4a96-9f42-0e82c0a3c4c1 admin admin] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:21:51.105033 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b347351-93f7-4a96-9f42-0e82c0a3c4c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:21:51.105454 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b347351-93f7-4a96-9f42-0e82c0a3c4c1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:21:51.105819 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b347351-93f7-4a96-9f42-0e82c0a3c4c1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:21:51.107738 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b347351-93f7-4a96-9f42-0e82c0a3c4c1 admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:21:51.232740 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1303/2627] 10.222.0.46 () {60 vars in 1153 bytes} [Mon Oct 10 12:21:51 2022] POST /identity/v3/projects => generated 387 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 10 12:21:51.236998 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7ebbaf-97e2-4467-b772-7ab939ce889f None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:21:51.248772 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7ebbaf-97e2-4467-b772-7ab939ce889f None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:21:51.262169 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:21:51.317281 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add26c03-2e79-46eb-982f-0335ef82ff7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:21:51.317895 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add26c03-2e79-46eb-982f-0335ef82ff7f admin admin] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:21:51.318168 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add26c03-2e79-46eb-982f-0335ef82ff7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:21:51.318442 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add26c03-2e79-46eb-982f-0335ef82ff7f admin admin] PATH_INFO: `/v3/users` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:21:51.319106 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add26c03-2e79-46eb-982f-0335ef82ff7f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:21:51.321007 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add26c03-2e79-46eb-982f-0335ef82ff7f admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:21:51.397304 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1325/2628] 10.222.0.46 () {60 vars in 1144 bytes} [Mon Oct 10 12:21:51 2022] POST /identity/v3/users => generated 294 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 12:21:51.401370 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b347351-93f7-4a96-9f42-0e82c0a3c4c1 admin admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:21:51.413208 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b347351-93f7-4a96-9f42-0e82c0a3c4c1 admin admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:21:51.417964 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d37638a-d4b1-48e5-a2aa-35001cdc3320 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:21:51.418569 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d37638a-d4b1-48e5-a2aa-35001cdc3320 admin admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:21:51.418888 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d37638a-d4b1-48e5-a2aa-35001cdc3320 admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:21:51.419168 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d37638a-d4b1-48e5-a2aa-35001cdc3320 admin admin] PATH_INFO: `/v3/roles` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:21:51.419816 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d37638a-d4b1-48e5-a2aa-35001cdc3320 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:21:51.421556 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d37638a-d4b1-48e5-a2aa-35001cdc3320 admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:21:51.431450 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1304/2629] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 10 12:21:51 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 12:21:51.435364 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add26c03-2e79-46eb-982f-0335ef82ff7f admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:21:51.445491 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add26c03-2e79-46eb-982f-0335ef82ff7f admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:21:51.450268 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc836ca5-c11c-41c7-88da-6bd94aac0417 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:21:51.452302 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc836ca5-c11c-41c7-88da-6bd94aac0417 admin admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:21:51.452302 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc836ca5-c11c-41c7-88da-6bd94aac0417 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:21:51.452302 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc836ca5-c11c-41c7-88da-6bd94aac0417 admin admin] PATH_INFO: `/v3/roles` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:21:51.452302 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc836ca5-c11c-41c7-88da-6bd94aac0417 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:21:51.454244 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc836ca5-c11c-41c7-88da-6bd94aac0417 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:21:51.465380 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1326/2630] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 10 12:21:51 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 12:21:51.470262 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d37638a-d4b1-48e5-a2aa-35001cdc3320 admin admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:21:51.480634 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d37638a-d4b1-48e5-a2aa-35001cdc3320 admin admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:21:51.486003 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6328a041-4fd1-4f87-9be3-80eb2db5084b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:21:51.486656 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6328a041-4fd1-4f87-9be3-80eb2db5084b admin admin] REQUEST_METHOD: `PUT` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:21:51.486969 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6328a041-4fd1-4f87-9be3-80eb2db5084b admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:21:51.487249 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6328a041-4fd1-4f87-9be3-80eb2db5084b admin admin] PATH_INFO: `/v3/projects/808a9a8554544780957e3d75f523de48/users/6fdc3a6422c04ddea95915a8de497c2c/roles/b2cb2a6e818e47338ce071e6fedfc673` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:21:51.502834 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6328a041-4fd1-4f87-9be3-80eb2db5084b admin admin] RBAC: Authorizing `identity:create_grant(project_id=808a9a8554544780957e3d75f523de48, user_id=6fdc3a6422c04ddea95915a8de497c2c, role_id=b2cb2a6e818e47338ce071e6fedfc673)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:21:51.505520 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6328a041-4fd1-4f87-9be3-80eb2db5084b admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:21:51.598951 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1305/2631] 10.222.0.46 () {60 vars in 1483 bytes} [Mon Oct 10 12:21:51 2022] PUT /identity/v3/projects/808a9a8554544780957e3d75f523de48/users/6fdc3a6422c04ddea95915a8de497c2c/roles/b2cb2a6e818e47338ce071e6fedfc673 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:21:51.609107 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10157de-8086-4c8d-95d4-0ab2d29097d1 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:21:51.609387 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10157de-8086-4c8d-95d4-0ab2d29097d1 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:21:51.609659 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10157de-8086-4c8d-95d4-0ab2d29097d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:21:51.638260 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-a10157de-8086-4c8d-95d4-0ab2d29097d1 None None] MFA Rules not processed for user `6fdc3a6422c04ddea95915a8de497c2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:21:51.639490 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-a10157de-8086-4c8d-95d4-0ab2d29097d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:21:51.702446 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1327/2632] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:21:51 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:21:51.707365 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6328a041-4fd1-4f87-9be3-80eb2db5084b admin admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:21:51.769359 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6328a041-4fd1-4f87-9be3-80eb2db5084b admin admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:21:51.774372 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d12503d-35b3-4770-903b-9747d84f0af5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:21:51.775135 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d12503d-35b3-4770-903b-9747d84f0af5 admin admin] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:21:51.775422 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d12503d-35b3-4770-903b-9747d84f0af5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:21:51.775698 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d12503d-35b3-4770-903b-9747d84f0af5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:21:51.776372 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d12503d-35b3-4770-903b-9747d84f0af5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:21:51.778298 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d12503d-35b3-4770-903b-9747d84f0af5 admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:21:51.859645 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1306/2633] 10.222.0.46 () {60 vars in 1153 bytes} [Mon Oct 10 12:21:51 2022] POST /identity/v3/projects => generated 385 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 12:21:51.864323 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10157de-8086-4c8d-95d4-0ab2d29097d1 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:21:51.876095 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10157de-8086-4c8d-95d4-0ab2d29097d1 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:21:51.935941 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fb6d3d-3193-4ac3-a43f-1395e8efdd82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:21:51.936555 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fb6d3d-3193-4ac3-a43f-1395e8efdd82 admin admin] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:21:51.936840 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fb6d3d-3193-4ac3-a43f-1395e8efdd82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:21:51.937116 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fb6d3d-3193-4ac3-a43f-1395e8efdd82 admin admin] PATH_INFO: `/v3/users` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:21:51.937778 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18fb6d3d-3193-4ac3-a43f-1395e8efdd82 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:21:51.939759 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18fb6d3d-3193-4ac3-a43f-1395e8efdd82 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:21:52.021819 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1328/2634] 10.222.0.46 () {60 vars in 1144 bytes} [Mon Oct 10 12:21:51 2022] POST /identity/v3/users => generated 299 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 12:21:52.025741 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d12503d-35b3-4770-903b-9747d84f0af5 admin admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:21:52.036637 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d12503d-35b3-4770-903b-9747d84f0af5 admin admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:21:52.041268 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b0281d-e6ef-4f9f-bd4f-0d2f27971a3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:21:52.041858 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b0281d-e6ef-4f9f-bd4f-0d2f27971a3e admin admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:21:52.042140 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b0281d-e6ef-4f9f-bd4f-0d2f27971a3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:21:52.042417 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b0281d-e6ef-4f9f-bd4f-0d2f27971a3e admin admin] PATH_INFO: `/v3/roles` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:21:52.043095 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b0281d-e6ef-4f9f-bd4f-0d2f27971a3e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:21:52.044695 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b0281d-e6ef-4f9f-bd4f-0d2f27971a3e admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:21:52.055882 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1307/2635] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 10 12:21:52 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 12:21:52.060058 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fb6d3d-3193-4ac3-a43f-1395e8efdd82 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:21:52.066478 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:21:52.070608 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fb6d3d-3193-4ac3-a43f-1395e8efdd82 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:21:52.075704 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6989d55d-6fc3-4cc5-a045-8abaef47cc78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:21:52.076398 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6989d55d-6fc3-4cc5-a045-8abaef47cc78 admin admin] REQUEST_METHOD: `PUT` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:21:52.076679 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6989d55d-6fc3-4cc5-a045-8abaef47cc78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:21:52.076956 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6989d55d-6fc3-4cc5-a045-8abaef47cc78 admin admin] PATH_INFO: `/v3/projects/0c75e6565ae6473c96760c4dd7b22e4c/users/912b41f3342d4035b552f8e0b5311f48/roles/0d822a0674b6495fac623877a3b38025` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:21:52.095040 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6989d55d-6fc3-4cc5-a045-8abaef47cc78 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0c75e6565ae6473c96760c4dd7b22e4c, user_id=912b41f3342d4035b552f8e0b5311f48, role_id=0d822a0674b6495fac623877a3b38025)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:21:52.098507 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6989d55d-6fc3-4cc5-a045-8abaef47cc78 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:21:52.197172 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1329/2636] 10.222.0.46 () {60 vars in 1483 bytes} [Mon Oct 10 12:21:52 2022] PUT /identity/v3/projects/0c75e6565ae6473c96760c4dd7b22e4c/users/912b41f3342d4035b552f8e0b5311f48/roles/0d822a0674b6495fac623877a3b38025 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:21:52.201240 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b0281d-e6ef-4f9f-bd4f-0d2f27971a3e admin admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:21:52.213779 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b0281d-e6ef-4f9f-bd4f-0d2f27971a3e admin admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:21:52.274823 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310a53a3-c5df-4d3a-9e60-f2b04111a15c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:21:52.275371 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310a53a3-c5df-4d3a-9e60-f2b04111a15c admin admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:21:52.275655 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310a53a3-c5df-4d3a-9e60-f2b04111a15c admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:21:52.275931 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310a53a3-c5df-4d3a-9e60-f2b04111a15c admin admin] PATH_INFO: `/v3/roles` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:21:52.276578 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310a53a3-c5df-4d3a-9e60-f2b04111a15c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:21:52.278256 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310a53a3-c5df-4d3a-9e60-f2b04111a15c admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:21:52.288927 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1308/2637] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 10 12:21:52 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 12:21:52.296165 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6989d55d-6fc3-4cc5-a045-8abaef47cc78 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:21:52.304805 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6989d55d-6fc3-4cc5-a045-8abaef47cc78 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:21:52.310341 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c0654f-7add-4ad9-ba13-99a685f7f2b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:21:52.311159 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c0654f-7add-4ad9-ba13-99a685f7f2b6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:21:52.311478 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c0654f-7add-4ad9-ba13-99a685f7f2b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:21:52.311804 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c0654f-7add-4ad9-ba13-99a685f7f2b6 admin admin] PATH_INFO: `/v3/projects/0c75e6565ae6473c96760c4dd7b22e4c/users/912b41f3342d4035b552f8e0b5311f48/roles/0d822a0674b6495fac623877a3b38025` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:21:52.314575 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c0654f-7add-4ad9-ba13-99a685f7f2b6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0c75e6565ae6473c96760c4dd7b22e4c, user_id=912b41f3342d4035b552f8e0b5311f48, role_id=0d822a0674b6495fac623877a3b38025)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:21:52.317545 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c0654f-7add-4ad9-ba13-99a685f7f2b6 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:21:52.330727 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1330/2638] 10.222.0.46 () {60 vars in 1483 bytes} [Mon Oct 10 12:21:52 2022] PUT /identity/v3/projects/0c75e6565ae6473c96760c4dd7b22e4c/users/912b41f3342d4035b552f8e0b5311f48/roles/0d822a0674b6495fac623877a3b38025 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 10 12:21:52.343866 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7305b63d-ee66-4c2e-a760-b29e2affbb19 None None] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:21:52.344207 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7305b63d-ee66-4c2e-a760-b29e2affbb19 None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:21:52.345215 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7305b63d-ee66-4c2e-a760-b29e2affbb19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:21:52.352922 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:21:52.377057 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.auth.core [None req-7305b63d-ee66-4c2e-a760-b29e2affbb19 None None] MFA Rules not processed for user `912b41f3342d4035b552f8e0b5311f48`. Rule list: `[]` (Enabled: `True`). {{(pid=81676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:21:52.378304 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-7305b63d-ee66-4c2e-a760-b29e2affbb19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:21:52.464770 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1309/2639] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:21:52 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:21:52.475809 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c0654f-7add-4ad9-ba13-99a685f7f2b6 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:21:52.477526 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-c7c0654f-7add-4ad9-ba13-99a685f7f2b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:21:52.541231 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c0654f-7add-4ad9-ba13-99a685f7f2b6 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:21:52.545280 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314f5621-d133-4a7b-943f-321b367c2875 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3ae093470484b93b23dcfae1657df9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad3772f7b6a47c28b7ba37a9316a3c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:21:52.545901 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314f5621-d133-4a7b-943f-321b367c2875 service cinder] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:21:52.546186 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314f5621-d133-4a7b-943f-321b367c2875 service cinder] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:21:52.546505 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314f5621-d133-4a7b-943f-321b367c2875 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:21:52.556957 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314f5621-d133-4a7b-943f-321b367c2875 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:21:52.558396 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314f5621-d133-4a7b-943f-321b367c2875 service cinder] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:21:52.568687 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1331/2640] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 10 12:21:52 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:21:52.623924 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7305b63d-ee66-4c2e-a760-b29e2affbb19 None None] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:21:52.625437 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-7305b63d-ee66-4c2e-a760-b29e2affbb19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:21:52.700712 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7305b63d-ee66-4c2e-a760-b29e2affbb19 None None] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:21:52.706077 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68be68f-be72-494b-a8ce-f7a0d8bffeba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:21:52.706932 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68be68f-be72-494b-a8ce-f7a0d8bffeba admin admin] REQUEST_METHOD: `DELETE` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:21:52.707165 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68be68f-be72-494b-a8ce-f7a0d8bffeba admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:21:52.707429 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68be68f-be72-494b-a8ce-f7a0d8bffeba admin admin] PATH_INFO: `/v3/users/eb72ac91a0b848c1914239e868e2a445` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:21:52.708569 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68be68f-be72-494b-a8ce-f7a0d8bffeba admin admin] RBAC: Authorizing `identity:delete_user(user_id=eb72ac91a0b848c1914239e868e2a445)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:21:52.710464 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68be68f-be72-494b-a8ce-f7a0d8bffeba admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:21:52.937222 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-a68be68f-be72-494b-a8ce-f7a0d8bffeba admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eb72ac91a0b848c1914239e868e2a445'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:21:52.945528 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-a68be68f-be72-494b-a8ce-f7a0d8bffeba admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eb72ac91a0b848c1914239e868e2a445'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:21:52.946562 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-a68be68f-be72-494b-a8ce-f7a0d8bffeba admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eb72ac91a0b848c1914239e868e2a445'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:21:53.027662 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-a68be68f-be72-494b-a8ce-f7a0d8bffeba admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eb72ac91a0b848c1914239e868e2a445'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:21:53.040825 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-a68be68f-be72-494b-a8ce-f7a0d8bffeba admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eb72ac91a0b848c1914239e868e2a445'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:21:53.047013 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1310/2641] 10.222.0.46 () {58 vars in 1224 bytes} [Mon Oct 10 12:21:52 2022] DELETE /identity/v3/users/eb72ac91a0b848c1914239e868e2a445 => generated 0 bytes in 425 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:21:53.052029 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8335d364-cb33-49c7-b986-9f09a5340086 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:21:53.052318 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8335d364-cb33-49c7-b986-9f09a5340086 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:21:53.052598 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8335d364-cb33-49c7-b986-9f09a5340086 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:21:53.072718 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-8335d364-cb33-49c7-b986-9f09a5340086 None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:21:53.074126 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-8335d364-cb33-49c7-b986-9f09a5340086 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:21:53.135340 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1332/2642] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:21:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:21:53.145312 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68be68f-be72-494b-a8ce-f7a0d8bffeba admin admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:21:53.149030 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-a68be68f-be72-494b-a8ce-f7a0d8bffeba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:21:53.163694 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:21:53.213549 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68be68f-be72-494b-a8ce-f7a0d8bffeba admin admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:21:53.217180 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4d4f21-089e-493b-9075-0b32efb67a93 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58b1c7c85302421cba823ace4201861f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad3772f7b6a47c28b7ba37a9316a3c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:21:53.217778 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4d4f21-089e-493b-9075-0b32efb67a93 service neutron] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:21:53.218054 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4d4f21-089e-493b-9075-0b32efb67a93 service neutron] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:21:53.218323 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4d4f21-089e-493b-9075-0b32efb67a93 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:21:53.227892 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4d4f21-089e-493b-9075-0b32efb67a93 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:21:53.229363 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4d4f21-089e-493b-9075-0b32efb67a93 service neutron] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:21:53.240236 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1311/2643] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 10 12:21:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:21:53.713311 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8335d364-cb33-49c7-b986-9f09a5340086 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:21:53.714994 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-8335d364-cb33-49c7-b986-9f09a5340086 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:21:53.724938 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4d4f21-089e-493b-9075-0b32efb67a93 service neutron] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:21:53.726470 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-ed4d4f21-089e-493b-9075-0b32efb67a93 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:21:53.728375 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8335d364-cb33-49c7-b986-9f09a5340086 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:21:53.735104 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc8229c-cc2c-41b9-abb2-e50cca6254a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:21:53.735846 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc8229c-cc2c-41b9-abb2-e50cca6254a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:21:53.736131 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc8229c-cc2c-41b9-abb2-e50cca6254a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:21:53.736406 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc8229c-cc2c-41b9-abb2-e50cca6254a0 admin admin] PATH_INFO: `/v3/projects/a037f5e5d96a48cb800baea601c9fb0f` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:21:53.739037 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc8229c-cc2c-41b9-abb2-e50cca6254a0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a037f5e5d96a48cb800baea601c9fb0f)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:21:53.741155 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc8229c-cc2c-41b9-abb2-e50cca6254a0 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:21:53.792375 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4d4f21-089e-493b-9075-0b32efb67a93 service neutron] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:21:53.795990 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77c19b4-dc89-45ca-a6ac-a10ae4657f07 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3ae093470484b93b23dcfae1657df9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad3772f7b6a47c28b7ba37a9316a3c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:21:53.796592 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77c19b4-dc89-45ca-a6ac-a10ae4657f07 service cinder] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:21:53.796870 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77c19b4-dc89-45ca-a6ac-a10ae4657f07 service cinder] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:21:53.797195 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77c19b4-dc89-45ca-a6ac-a10ae4657f07 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:21:53.799225 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-c77c19b4-dc89-45ca-a6ac-a10ae4657f07 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:21:53.855113 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-4bc8229c-cc2c-41b9-abb2-e50cca6254a0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a037f5e5d96a48cb800baea601c9fb0f', 'request_id': 'req-4bc8229c-cc2c-41b9-abb2-e50cca6254a0'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:21:53.859846 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77c19b4-dc89-45ca-a6ac-a10ae4657f07 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:21:53.861291 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77c19b4-dc89-45ca-a6ac-a10ae4657f07 service cinder] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:21:53.861430 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-4bc8229c-cc2c-41b9-abb2-e50cca6254a0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a037f5e5d96a48cb800baea601c9fb0f', 'request_id': 'req-4bc8229c-cc2c-41b9-abb2-e50cca6254a0'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:21:53.867657 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-4bc8229c-cc2c-41b9-abb2-e50cca6254a0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a037f5e5d96a48cb800baea601c9fb0f', 'request_id': 'req-4bc8229c-cc2c-41b9-abb2-e50cca6254a0'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:21:53.870428 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=91,keystone:INSERT=3,keystone:DELETE=6 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:21:53.871093 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1312/2644] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 10 12:21:53 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:21:53.952912 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-4bc8229c-cc2c-41b9-abb2-e50cca6254a0 admin admin] The token cache is being invalidate because project a037f5e5d96a48cb800baea601c9fb0f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81677) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 10 12:21:53.953203 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-4bc8229c-cc2c-41b9-abb2-e50cca6254a0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:21:53.954496 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-4bc8229c-cc2c-41b9-abb2-e50cca6254a0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:21:53.956436 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1333/2645] 10.222.0.46 () {58 vars in 1233 bytes} [Mon Oct 10 12:21:53 2022] DELETE /identity/v3/projects/a037f5e5d96a48cb800baea601c9fb0f => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:21:53.961342 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d913cb-16d8-425a-a75c-61ab1b108d15 None None] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:21:53.961630 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d913cb-16d8-425a-a75c-61ab1b108d15 None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:21:53.961910 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d913cb-16d8-425a-a75c-61ab1b108d15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:21:53.981811 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.auth.core [None req-52d913cb-16d8-425a-a75c-61ab1b108d15 None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:21:53.983046 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-52d913cb-16d8-425a-a75c-61ab1b108d15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:21:54.044584 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1313/2646] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:21:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 10 12:21:54.052461 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c21dc8a-aaa5-4582-8854-002cc2d0a548 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:21:54.052741 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c21dc8a-aaa5-4582-8854-002cc2d0a548 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:21:54.053018 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c21dc8a-aaa5-4582-8854-002cc2d0a548 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:21:54.074777 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-5c21dc8a-aaa5-4582-8854-002cc2d0a548 None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:21:54.076065 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-5c21dc8a-aaa5-4582-8854-002cc2d0a548 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:21:54.084629 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1334/2647] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:21:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:21:54.089243 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d913cb-16d8-425a-a75c-61ab1b108d15 None None] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:21:54.095599 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:21:54.104102 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d913cb-16d8-425a-a75c-61ab1b108d15 None None] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:21:54.104701 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7c70f2-969b-4c2d-9c23-2a3f4f6edebd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:21:54.105441 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7c70f2-969b-4c2d-9c23-2a3f4f6edebd admin admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:21:54.105656 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7c70f2-969b-4c2d-9c23-2a3f4f6edebd admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:21:54.105935 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7c70f2-969b-4c2d-9c23-2a3f4f6edebd admin admin] PATH_INFO: `/v3/domains` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:21:54.106594 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7c70f2-969b-4c2d-9c23-2a3f4f6edebd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 12:21:54.106941 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7c70f2-969b-4c2d-9c23-2a3f4f6edebd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:21:54.108545 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7c70f2-969b-4c2d-9c23-2a3f4f6edebd admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:21:54.120376 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1314/2648] 10.222.0.46 () {58 vars in 1153 bytes} [Mon Oct 10 12:21:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 12:21:54.124840 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c21dc8a-aaa5-4582-8854-002cc2d0a548 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:21:54.134764 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c21dc8a-aaa5-4582-8854-002cc2d0a548 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:21:54.139708 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3690b8c5-a98b-4a36-a929-52d64faad674 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:21:54.140335 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3690b8c5-a98b-4a36-a929-52d64faad674 admin admin] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:21:54.140611 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3690b8c5-a98b-4a36-a929-52d64faad674 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:21:54.140927 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3690b8c5-a98b-4a36-a929-52d64faad674 admin admin] PATH_INFO: `/v3/projects` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:21:54.142186 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3690b8c5-a98b-4a36-a929-52d64faad674 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:21:54.144082 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3690b8c5-a98b-4a36-a929-52d64faad674 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:21:54.246055 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1335/2649] 10.222.0.46 () {60 vars in 1153 bytes} [Mon Oct 10 12:21:54 2022] POST /identity/v3/projects => generated 385 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 10 12:21:54.250430 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7c70f2-969b-4c2d-9c23-2a3f4f6edebd admin admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:21:54.261772 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7c70f2-969b-4c2d-9c23-2a3f4f6edebd admin admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:21:54.323421 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc2758e-05f7-4e99-bc91-a5534b3cb7be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:21:54.324043 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc2758e-05f7-4e99-bc91-a5534b3cb7be admin admin] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:21:54.324325 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc2758e-05f7-4e99-bc91-a5534b3cb7be admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:21:54.324601 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc2758e-05f7-4e99-bc91-a5534b3cb7be admin admin] PATH_INFO: `/v3/users` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:21:54.325249 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc2758e-05f7-4e99-bc91-a5534b3cb7be admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:21:54.327196 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc2758e-05f7-4e99-bc91-a5534b3cb7be admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:21:54.423170 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1315/2650] 10.222.0.46 () {60 vars in 1144 bytes} [Mon Oct 10 12:21:54 2022] POST /identity/v3/users => generated 293 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 10 12:21:54.427063 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3690b8c5-a98b-4a36-a929-52d64faad674 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:21:54.437792 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3690b8c5-a98b-4a36-a929-52d64faad674 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:21:54.443359 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c82c6df-6876-45cc-9054-03af5ccbb024 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:21:54.444002 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c82c6df-6876-45cc-9054-03af5ccbb024 admin admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:21:54.445539 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c82c6df-6876-45cc-9054-03af5ccbb024 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:21:54.445539 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c82c6df-6876-45cc-9054-03af5ccbb024 admin admin] PATH_INFO: `/v3/roles` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:21:54.445539 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c82c6df-6876-45cc-9054-03af5ccbb024 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:21:54.446813 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c82c6df-6876-45cc-9054-03af5ccbb024 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:21:54.458006 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1336/2651] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 10 12:21:54 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 12:21:54.466726 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc2758e-05f7-4e99-bc91-a5534b3cb7be admin admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:21:54.482766 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc2758e-05f7-4e99-bc91-a5534b3cb7be admin admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:21:54.489587 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756d7a96-a7be-4209-9546-c9e63f6cd154 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:21:54.490205 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756d7a96-a7be-4209-9546-c9e63f6cd154 admin admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:21:54.490502 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756d7a96-a7be-4209-9546-c9e63f6cd154 admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:21:54.490825 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756d7a96-a7be-4209-9546-c9e63f6cd154 admin admin] PATH_INFO: `/v3/roles` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:21:54.491428 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756d7a96-a7be-4209-9546-c9e63f6cd154 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:21:54.493049 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756d7a96-a7be-4209-9546-c9e63f6cd154 admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:21:54.510025 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1316/2652] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 10 12:21:54 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 12:21:54.511359 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c82c6df-6876-45cc-9054-03af5ccbb024 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:21:54.524463 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c82c6df-6876-45cc-9054-03af5ccbb024 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:21:54.529500 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803074e5-7b53-423f-a763-d3d10c9a6cf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:21:54.530260 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803074e5-7b53-423f-a763-d3d10c9a6cf9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:21:54.530548 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803074e5-7b53-423f-a763-d3d10c9a6cf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:21:54.530888 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803074e5-7b53-423f-a763-d3d10c9a6cf9 admin admin] PATH_INFO: `/v3/projects/cf6abc8a3f5b4648b470cc79266e4780/users/0f2b6e2a57dc4e699ee63f624666bf6b/roles/b2cb2a6e818e47338ce071e6fedfc673` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:21:54.548713 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803074e5-7b53-423f-a763-d3d10c9a6cf9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cf6abc8a3f5b4648b470cc79266e4780, user_id=0f2b6e2a57dc4e699ee63f624666bf6b, role_id=b2cb2a6e818e47338ce071e6fedfc673)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:21:54.551575 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803074e5-7b53-423f-a763-d3d10c9a6cf9 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:21:54.664678 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1337/2653] 10.222.0.46 () {60 vars in 1483 bytes} [Mon Oct 10 12:21:54 2022] PUT /identity/v3/projects/cf6abc8a3f5b4648b470cc79266e4780/users/0f2b6e2a57dc4e699ee63f624666bf6b/roles/b2cb2a6e818e47338ce071e6fedfc673 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:21:54.666481 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd0c4bf-751b-47b0-abd3-6256a023cd38 None None] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:21:54.666892 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd0c4bf-751b-47b0-abd3-6256a023cd38 None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:21:54.667236 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd0c4bf-751b-47b0-abd3-6256a023cd38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:21:54.697838 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.auth.core [None req-3dd0c4bf-751b-47b0-abd3-6256a023cd38 None None] MFA Rules not processed for user `0f2b6e2a57dc4e699ee63f624666bf6b`. Rule list: `[]` (Enabled: `True`). {{(pid=81676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:21:54.699227 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-3dd0c4bf-751b-47b0-abd3-6256a023cd38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:21:54.769696 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1317/2654] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:21:54 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:21:54.781225 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803074e5-7b53-423f-a763-d3d10c9a6cf9 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:21:54.783766 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-803074e5-7b53-423f-a763-d3d10c9a6cf9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:21:54.848540 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803074e5-7b53-423f-a763-d3d10c9a6cf9 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:21:54.852442 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee513ef3-3f72-41e8-a3aa-9cb84fea68cc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3ae093470484b93b23dcfae1657df9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad3772f7b6a47c28b7ba37a9316a3c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:21:54.853147 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee513ef3-3f72-41e8-a3aa-9cb84fea68cc service cinder] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:21:54.853452 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee513ef3-3f72-41e8-a3aa-9cb84fea68cc service cinder] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:21:54.853727 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee513ef3-3f72-41e8-a3aa-9cb84fea68cc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:21:54.863769 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee513ef3-3f72-41e8-a3aa-9cb84fea68cc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:21:54.866027 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee513ef3-3f72-41e8-a3aa-9cb84fea68cc service cinder] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:21:54.875461 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1338/2655] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 10 12:21:54 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:21:55.709568 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd0c4bf-751b-47b0-abd3-6256a023cd38 None None] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:21:55.711222 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-3dd0c4bf-751b-47b0-abd3-6256a023cd38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:21:55.785350 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd0c4bf-751b-47b0-abd3-6256a023cd38 None None] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:21:55.789934 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cebb86-643b-4f73-ba9d-45d1fe29a3ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:21:55.790598 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cebb86-643b-4f73-ba9d-45d1fe29a3ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:21:55.790905 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cebb86-643b-4f73-ba9d-45d1fe29a3ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:21:55.791183 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cebb86-643b-4f73-ba9d-45d1fe29a3ff admin admin] PATH_INFO: `/v3/users/ab50583207134c8fa2026cc245d252c2` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:21:55.792364 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4cebb86-643b-4f73-ba9d-45d1fe29a3ff admin admin] RBAC: Authorizing `identity:delete_user(user_id=ab50583207134c8fa2026cc245d252c2)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:21:55.795614 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4cebb86-643b-4f73-ba9d-45d1fe29a3ff admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:21:55.797222 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:21:56.028093 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-f4cebb86-643b-4f73-ba9d-45d1fe29a3ff admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ab50583207134c8fa2026cc245d252c2'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:21:56.038824 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-f4cebb86-643b-4f73-ba9d-45d1fe29a3ff admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ab50583207134c8fa2026cc245d252c2'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:21:56.039584 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-f4cebb86-643b-4f73-ba9d-45d1fe29a3ff admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ab50583207134c8fa2026cc245d252c2'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:21:56.096746 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-f4cebb86-643b-4f73-ba9d-45d1fe29a3ff admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ab50583207134c8fa2026cc245d252c2'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:21:56.111313 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-f4cebb86-643b-4f73-ba9d-45d1fe29a3ff admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ab50583207134c8fa2026cc245d252c2'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:21:56.117922 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1318/2656] 10.222.0.46 () {58 vars in 1224 bytes} [Mon Oct 10 12:21:55 2022] DELETE /identity/v3/users/ab50583207134c8fa2026cc245d252c2 => generated 0 bytes in 411 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:21:56.122424 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d060dc51-3d1b-4845-9687-92f18f96dc59 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:21:56.122960 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d060dc51-3d1b-4845-9687-92f18f96dc59 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:21:56.123126 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d060dc51-3d1b-4845-9687-92f18f96dc59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:21:56.142910 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-d060dc51-3d1b-4845-9687-92f18f96dc59 None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:21:56.144578 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-d060dc51-3d1b-4845-9687-92f18f96dc59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:21:56.207091 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1339/2657] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:21:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:21:56.217571 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cebb86-643b-4f73-ba9d-45d1fe29a3ff admin admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:21:56.219098 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-f4cebb86-643b-4f73-ba9d-45d1fe29a3ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:21:56.281928 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cebb86-643b-4f73-ba9d-45d1fe29a3ff admin admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:21:56.286067 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe14f86-e861-4d5d-802a-92c15e1f0115 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58b1c7c85302421cba823ace4201861f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad3772f7b6a47c28b7ba37a9316a3c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:21:56.286764 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe14f86-e861-4d5d-802a-92c15e1f0115 service neutron] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:21:56.287138 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe14f86-e861-4d5d-802a-92c15e1f0115 service neutron] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:21:56.287479 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe14f86-e861-4d5d-802a-92c15e1f0115 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:21:56.297798 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe14f86-e861-4d5d-802a-92c15e1f0115 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:21:56.299287 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe14f86-e861-4d5d-802a-92c15e1f0115 service neutron] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:21:56.309395 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1319/2658] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 10 12:21:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 10 12:21:56.775377 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d060dc51-3d1b-4845-9687-92f18f96dc59 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:21:56.777312 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-d060dc51-3d1b-4845-9687-92f18f96dc59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:21:56.796125 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d060dc51-3d1b-4845-9687-92f18f96dc59 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:21:56.803179 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f9bec8-8591-4f1b-b83e-e7b85c05074d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:21:56.807829 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f9bec8-8591-4f1b-b83e-e7b85c05074d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:21:56.807829 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f9bec8-8591-4f1b-b83e-e7b85c05074d admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:21:56.807829 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f9bec8-8591-4f1b-b83e-e7b85c05074d admin admin] PATH_INFO: `/v3/projects/4bd4f7bcd357441185ee61871404c84b` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:21:56.807829 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f9bec8-8591-4f1b-b83e-e7b85c05074d admin admin] RBAC: Authorizing `identity:delete_project(project_id=4bd4f7bcd357441185ee61871404c84b)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:21:56.807829 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f9bec8-8591-4f1b-b83e-e7b85c05074d admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:21:56.919626 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:DELETE=3,keystone:SELECT=95,keystone:INSERT=2 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:21:56.935324 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-23f9bec8-8591-4f1b-b83e-e7b85c05074d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4bd4f7bcd357441185ee61871404c84b', 'request_id': 'req-23f9bec8-8591-4f1b-b83e-e7b85c05074d'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:21:56.941364 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-23f9bec8-8591-4f1b-b83e-e7b85c05074d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4bd4f7bcd357441185ee61871404c84b', 'request_id': 'req-23f9bec8-8591-4f1b-b83e-e7b85c05074d'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:21:56.947578 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-23f9bec8-8591-4f1b-b83e-e7b85c05074d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4bd4f7bcd357441185ee61871404c84b', 'request_id': 'req-23f9bec8-8591-4f1b-b83e-e7b85c05074d'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:21:57.033817 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-23f9bec8-8591-4f1b-b83e-e7b85c05074d admin admin] The token cache is being invalidate because project 4bd4f7bcd357441185ee61871404c84b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81677) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 10 12:21:57.034203 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-23f9bec8-8591-4f1b-b83e-e7b85c05074d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:21:57.034874 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-23f9bec8-8591-4f1b-b83e-e7b85c05074d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:21:57.036629 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1340/2659] 10.222.0.46 () {58 vars in 1233 bytes} [Mon Oct 10 12:21:56 2022] DELETE /identity/v3/projects/4bd4f7bcd357441185ee61871404c84b => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 10 12:21:57.041603 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3810cd6-8335-4a0a-9be8-b49eaaa0c687 None None] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:21:57.042100 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3810cd6-8335-4a0a-9be8-b49eaaa0c687 None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:21:57.042198 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3810cd6-8335-4a0a-9be8-b49eaaa0c687 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:21:57.068629 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.auth.core [None req-f3810cd6-8335-4a0a-9be8-b49eaaa0c687 None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:21:57.069769 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-f3810cd6-8335-4a0a-9be8-b49eaaa0c687 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:21:57.129467 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1320/2660] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:21:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 10 12:21:57.137761 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad296d8d-04dc-4ce6-ad87-60e92d12d9e8 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:21:57.138295 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad296d8d-04dc-4ce6-ad87-60e92d12d9e8 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:21:57.138427 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad296d8d-04dc-4ce6-ad87-60e92d12d9e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:21:57.159873 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-ad296d8d-04dc-4ce6-ad87-60e92d12d9e8 None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:21:57.161921 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-ad296d8d-04dc-4ce6-ad87-60e92d12d9e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:21:57.169829 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1341/2661] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:21:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:21:57.174386 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3810cd6-8335-4a0a-9be8-b49eaaa0c687 None None] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:21:57.184323 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3810cd6-8335-4a0a-9be8-b49eaaa0c687 None None] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:21:57.189429 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6813025-893f-4e43-bf83-d3f8b7d0f84b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:21:57.190021 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6813025-893f-4e43-bf83-d3f8b7d0f84b admin admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:21:57.190295 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6813025-893f-4e43-bf83-d3f8b7d0f84b admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:21:57.190565 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6813025-893f-4e43-bf83-d3f8b7d0f84b admin admin] PATH_INFO: `/v3/domains` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:21:57.191284 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6813025-893f-4e43-bf83-d3f8b7d0f84b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 12:21:57.191582 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6813025-893f-4e43-bf83-d3f8b7d0f84b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:21:57.193150 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6813025-893f-4e43-bf83-d3f8b7d0f84b admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:21:57.205348 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1321/2662] 10.222.0.46 () {58 vars in 1153 bytes} [Mon Oct 10 12:21:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 12:21:57.209810 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad296d8d-04dc-4ce6-ad87-60e92d12d9e8 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:21:57.219343 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad296d8d-04dc-4ce6-ad87-60e92d12d9e8 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:21:57.224042 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a21627-4626-4b5a-821f-d237227b3dae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:21:57.224794 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a21627-4626-4b5a-821f-d237227b3dae admin admin] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:21:57.224938 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a21627-4626-4b5a-821f-d237227b3dae admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:21:57.225235 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a21627-4626-4b5a-821f-d237227b3dae admin admin] PATH_INFO: `/v3/projects` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:21:57.225824 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a21627-4626-4b5a-821f-d237227b3dae admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:21:57.227786 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a21627-4626-4b5a-821f-d237227b3dae admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:21:57.325972 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1342/2663] 10.222.0.46 () {60 vars in 1153 bytes} [Mon Oct 10 12:21:57 2022] POST /identity/v3/projects => generated 367 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 12:21:57.327955 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6813025-893f-4e43-bf83-d3f8b7d0f84b admin admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:21:57.338316 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6813025-893f-4e43-bf83-d3f8b7d0f84b admin admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:21:57.343621 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:21:57.396565 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98603c8-aa10-434a-9e9a-a36139ead34f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:21:57.397187 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98603c8-aa10-434a-9e9a-a36139ead34f admin admin] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:21:57.397469 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98603c8-aa10-434a-9e9a-a36139ead34f admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:21:57.397747 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98603c8-aa10-434a-9e9a-a36139ead34f admin admin] PATH_INFO: `/v3/users` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:21:57.398384 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98603c8-aa10-434a-9e9a-a36139ead34f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:21:57.400437 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98603c8-aa10-434a-9e9a-a36139ead34f admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:21:57.493809 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1322/2664] 10.222.0.46 () {60 vars in 1144 bytes} [Mon Oct 10 12:21:57 2022] POST /identity/v3/users => generated 284 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 12:21:57.498053 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a21627-4626-4b5a-821f-d237227b3dae admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:21:57.510141 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a21627-4626-4b5a-821f-d237227b3dae admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:21:57.515315 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515c37a6-f4ef-49dc-b0d7-e17cd8c954a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:21:57.516341 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515c37a6-f4ef-49dc-b0d7-e17cd8c954a4 admin admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:21:57.516341 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515c37a6-f4ef-49dc-b0d7-e17cd8c954a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:21:57.516559 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515c37a6-f4ef-49dc-b0d7-e17cd8c954a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:21:57.520186 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515c37a6-f4ef-49dc-b0d7-e17cd8c954a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:21:57.520186 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515c37a6-f4ef-49dc-b0d7-e17cd8c954a4 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:21:57.530079 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1343/2665] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 10 12:21:57 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 12:21:57.534238 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98603c8-aa10-434a-9e9a-a36139ead34f admin admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:21:57.544473 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98603c8-aa10-434a-9e9a-a36139ead34f admin admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:21:57.550123 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b237be54-b882-450c-9efa-b3f59f1e1425 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:21:57.550798 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b237be54-b882-450c-9efa-b3f59f1e1425 admin admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:21:57.551088 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b237be54-b882-450c-9efa-b3f59f1e1425 admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:21:57.551362 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b237be54-b882-450c-9efa-b3f59f1e1425 admin admin] PATH_INFO: `/v3/roles` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:21:57.552026 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b237be54-b882-450c-9efa-b3f59f1e1425 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:21:57.553853 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b237be54-b882-450c-9efa-b3f59f1e1425 admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:21:57.565282 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1323/2666] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 10 12:21:57 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 10 12:21:57.569733 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515c37a6-f4ef-49dc-b0d7-e17cd8c954a4 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:21:57.583251 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515c37a6-f4ef-49dc-b0d7-e17cd8c954a4 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:21:57.586251 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c16eada-a777-4a06-aa16-ea42483bfc05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:21:57.586952 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c16eada-a777-4a06-aa16-ea42483bfc05 admin admin] REQUEST_METHOD: `PUT` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:21:57.587234 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c16eada-a777-4a06-aa16-ea42483bfc05 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:21:57.587517 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c16eada-a777-4a06-aa16-ea42483bfc05 admin admin] PATH_INFO: `/v3/projects/2422eeb354644a5caa84292ee7b8b9f6/users/bbdf4d1c31fa4d3fa46a1e19fd422058/roles/b2cb2a6e818e47338ce071e6fedfc673` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:21:57.602903 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c16eada-a777-4a06-aa16-ea42483bfc05 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2422eeb354644a5caa84292ee7b8b9f6, user_id=bbdf4d1c31fa4d3fa46a1e19fd422058, role_id=b2cb2a6e818e47338ce071e6fedfc673)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:21:57.605638 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c16eada-a777-4a06-aa16-ea42483bfc05 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:21:57.700319 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1344/2667] 10.222.0.46 () {60 vars in 1483 bytes} [Mon Oct 10 12:21:57 2022] PUT /identity/v3/projects/2422eeb354644a5caa84292ee7b8b9f6/users/bbdf4d1c31fa4d3fa46a1e19fd422058/roles/b2cb2a6e818e47338ce071e6fedfc673 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:21:57.712217 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b7879a-9355-4ed3-83af-ee92463bb08b None None] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:21:57.712503 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b7879a-9355-4ed3-83af-ee92463bb08b None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:21:57.712778 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b7879a-9355-4ed3-83af-ee92463bb08b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:21:57.746200 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.auth.core [None req-23b7879a-9355-4ed3-83af-ee92463bb08b None None] MFA Rules not processed for user `bbdf4d1c31fa4d3fa46a1e19fd422058`. Rule list: `[]` (Enabled: `True`). {{(pid=81676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:21:57.747565 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-23b7879a-9355-4ed3-83af-ee92463bb08b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:21:57.822812 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1324/2668] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:21:57 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:21:57.839211 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c16eada-a777-4a06-aa16-ea42483bfc05 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:21:57.839211 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-1c16eada-a777-4a06-aa16-ea42483bfc05 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:21:57.920951 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c16eada-a777-4a06-aa16-ea42483bfc05 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:21:57.925427 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956436ab-9838-4d95-8e86-85f0261d24b5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3ae093470484b93b23dcfae1657df9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad3772f7b6a47c28b7ba37a9316a3c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:21:57.926159 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956436ab-9838-4d95-8e86-85f0261d24b5 service cinder] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:21:57.926464 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956436ab-9838-4d95-8e86-85f0261d24b5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:21:57.926842 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956436ab-9838-4d95-8e86-85f0261d24b5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:21:57.942827 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956436ab-9838-4d95-8e86-85f0261d24b5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:21:57.944874 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956436ab-9838-4d95-8e86-85f0261d24b5 service cinder] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:21:57.956955 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1345/2669] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 10 12:21:57 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 10 12:21:59.362024 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b7879a-9355-4ed3-83af-ee92463bb08b None None] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:21:59.363411 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-23b7879a-9355-4ed3-83af-ee92463bb08b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:21:59.427395 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b7879a-9355-4ed3-83af-ee92463bb08b None None] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:21:59.431947 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91663c0c-d7ec-4b7d-86df-3449ed2e347f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:21:59.432579 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91663c0c-d7ec-4b7d-86df-3449ed2e347f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:21:59.432862 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91663c0c-d7ec-4b7d-86df-3449ed2e347f admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:21:59.433140 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91663c0c-d7ec-4b7d-86df-3449ed2e347f admin admin] PATH_INFO: `/v3/users/0f2b6e2a57dc4e699ee63f624666bf6b` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:21:59.434295 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91663c0c-d7ec-4b7d-86df-3449ed2e347f admin admin] RBAC: Authorizing `identity:delete_user(user_id=0f2b6e2a57dc4e699ee63f624666bf6b)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:21:59.436248 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91663c0c-d7ec-4b7d-86df-3449ed2e347f admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:21:59.458930 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=95,keystone:INSERT=3,keystone:DELETE=2 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:21:59.737352 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-91663c0c-d7ec-4b7d-86df-3449ed2e347f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0f2b6e2a57dc4e699ee63f624666bf6b'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:21:59.744337 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-91663c0c-d7ec-4b7d-86df-3449ed2e347f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0f2b6e2a57dc4e699ee63f624666bf6b'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:21:59.745031 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-91663c0c-d7ec-4b7d-86df-3449ed2e347f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0f2b6e2a57dc4e699ee63f624666bf6b'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:21:59.822141 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-91663c0c-d7ec-4b7d-86df-3449ed2e347f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0f2b6e2a57dc4e699ee63f624666bf6b'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:21:59.835449 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-91663c0c-d7ec-4b7d-86df-3449ed2e347f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0f2b6e2a57dc4e699ee63f624666bf6b'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:21:59.842118 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1325/2670] 10.222.0.46 () {58 vars in 1224 bytes} [Mon Oct 10 12:21:59 2022] DELETE /identity/v3/users/0f2b6e2a57dc4e699ee63f624666bf6b => generated 0 bytes in 481 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:21:59.846745 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c96263c-b80a-4ccd-909e-915c880d1367 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:21:59.847251 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c96263c-b80a-4ccd-909e-915c880d1367 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:21:59.847349 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c96263c-b80a-4ccd-909e-915c880d1367 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:21:59.868586 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-5c96263c-b80a-4ccd-909e-915c880d1367 None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:21:59.869184 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-5c96263c-b80a-4ccd-909e-915c880d1367 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:21:59.917182 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:21:59.930014 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1346/2671] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:21:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:21:59.940286 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91663c0c-d7ec-4b7d-86df-3449ed2e347f admin admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:21:59.941706 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-91663c0c-d7ec-4b7d-86df-3449ed2e347f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:22:00.015956 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91663c0c-d7ec-4b7d-86df-3449ed2e347f admin admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:22:00.021610 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54903a46-ec80-49bf-8e90-c7e9650a288f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58b1c7c85302421cba823ace4201861f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad3772f7b6a47c28b7ba37a9316a3c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:22:00.022441 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54903a46-ec80-49bf-8e90-c7e9650a288f service neutron] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:22:00.022943 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54903a46-ec80-49bf-8e90-c7e9650a288f service neutron] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:22:00.023409 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54903a46-ec80-49bf-8e90-c7e9650a288f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:22:00.039460 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54903a46-ec80-49bf-8e90-c7e9650a288f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:22:00.041087 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54903a46-ec80-49bf-8e90-c7e9650a288f service neutron] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:22:00.052429 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1326/2672] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 10 12:21:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:22:00.512458 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c96263c-b80a-4ccd-909e-915c880d1367 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:22:00.513978 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-5c96263c-b80a-4ccd-909e-915c880d1367 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:22:00.529655 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c96263c-b80a-4ccd-909e-915c880d1367 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:22:00.535499 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c29964-ba89-4c60-af3a-9ac92500bfe0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:22:00.536379 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c29964-ba89-4c60-af3a-9ac92500bfe0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:22:00.537190 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c29964-ba89-4c60-af3a-9ac92500bfe0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:22:00.537190 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c29964-ba89-4c60-af3a-9ac92500bfe0 admin admin] PATH_INFO: `/v3/projects/cf6abc8a3f5b4648b470cc79266e4780` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:22:00.538093 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c29964-ba89-4c60-af3a-9ac92500bfe0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cf6abc8a3f5b4648b470cc79266e4780)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:22:00.540067 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c29964-ba89-4c60-af3a-9ac92500bfe0 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:22:00.668542 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-02c29964-ba89-4c60-af3a-9ac92500bfe0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cf6abc8a3f5b4648b470cc79266e4780', 'request_id': 'req-02c29964-ba89-4c60-af3a-9ac92500bfe0'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:22:00.674228 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-02c29964-ba89-4c60-af3a-9ac92500bfe0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cf6abc8a3f5b4648b470cc79266e4780', 'request_id': 'req-02c29964-ba89-4c60-af3a-9ac92500bfe0'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:22:00.681749 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-02c29964-ba89-4c60-af3a-9ac92500bfe0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cf6abc8a3f5b4648b470cc79266e4780', 'request_id': 'req-02c29964-ba89-4c60-af3a-9ac92500bfe0'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:22:00.746897 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-02c29964-ba89-4c60-af3a-9ac92500bfe0 admin admin] The token cache is being invalidate because project cf6abc8a3f5b4648b470cc79266e4780 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81677) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 10 12:22:00.747193 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-02c29964-ba89-4c60-af3a-9ac92500bfe0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:22:00.747834 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-02c29964-ba89-4c60-af3a-9ac92500bfe0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:22:00.749368 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1347/2673] 10.222.0.46 () {58 vars in 1233 bytes} [Mon Oct 10 12:22:00 2022] DELETE /identity/v3/projects/cf6abc8a3f5b4648b470cc79266e4780 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:22:00.754270 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f90e829-0874-4cdb-b694-24dd5d33a1aa None None] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:22:00.754775 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f90e829-0874-4cdb-b694-24dd5d33a1aa None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:22:00.754986 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f90e829-0874-4cdb-b694-24dd5d33a1aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:22:00.778950 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.auth.core [None req-0f90e829-0874-4cdb-b694-24dd5d33a1aa None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:22:00.780260 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-0f90e829-0874-4cdb-b694-24dd5d33a1aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:22:00.842354 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1327/2674] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:22:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:22:00.911754 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c43dbe-e779-4e5f-8811-da22d28aaa14 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:22:00.912344 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c43dbe-e779-4e5f-8811-da22d28aaa14 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:22:00.912344 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c43dbe-e779-4e5f-8811-da22d28aaa14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:22:00.934766 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-03c43dbe-e779-4e5f-8811-da22d28aaa14 None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:22:00.936061 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-03c43dbe-e779-4e5f-8811-da22d28aaa14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:22:00.943977 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1348/2675] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:22:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:22:00.948314 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f90e829-0874-4cdb-b694-24dd5d33a1aa None None] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:22:00.958224 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f90e829-0874-4cdb-b694-24dd5d33a1aa None None] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:22:00.963248 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7bdda28-e85d-4248-af36-6a979d885a6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:22:00.963859 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bdda28-e85d-4248-af36-6a979d885a6f admin admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:22:00.964137 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bdda28-e85d-4248-af36-6a979d885a6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:22:00.964401 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bdda28-e85d-4248-af36-6a979d885a6f admin admin] PATH_INFO: `/v3/domains` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:22:00.965047 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7bdda28-e85d-4248-af36-6a979d885a6f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 12:22:00.965344 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7bdda28-e85d-4248-af36-6a979d885a6f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:22:00.966997 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7bdda28-e85d-4248-af36-6a979d885a6f admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:22:00.979845 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1328/2676] 10.222.0.46 () {58 vars in 1153 bytes} [Mon Oct 10 12:22:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 12:22:00.984428 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c43dbe-e779-4e5f-8811-da22d28aaa14 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:22:00.994479 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c43dbe-e779-4e5f-8811-da22d28aaa14 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:22:00.999191 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced3c156-f5fb-4e97-9de6-a5411875fa0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:22:00.999798 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced3c156-f5fb-4e97-9de6-a5411875fa0d admin admin] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:22:01.000081 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced3c156-f5fb-4e97-9de6-a5411875fa0d admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:22:01.000358 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced3c156-f5fb-4e97-9de6-a5411875fa0d admin admin] PATH_INFO: `/v3/projects` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:22:01.001000 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced3c156-f5fb-4e97-9de6-a5411875fa0d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:22:01.002950 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced3c156-f5fb-4e97-9de6-a5411875fa0d admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:22:01.090117 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1349/2677] 10.222.0.46 () {60 vars in 1153 bytes} [Mon Oct 10 12:22:00 2022] POST /identity/v3/projects => generated 373 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 12:22:01.091286 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7bdda28-e85d-4248-af36-6a979d885a6f admin admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:22:01.101526 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7bdda28-e85d-4248-af36-6a979d885a6f admin admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:22:01.127825 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:DELETE=8,keystone:SELECT=91,keystone:INSERT=1 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:22:01.165910 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4142f7-adff-47cd-8c8a-b86cd735b8f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:22:01.166533 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4142f7-adff-47cd-8c8a-b86cd735b8f9 admin admin] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:22:01.166845 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4142f7-adff-47cd-8c8a-b86cd735b8f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:22:01.167164 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4142f7-adff-47cd-8c8a-b86cd735b8f9 admin admin] PATH_INFO: `/v3/users` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:22:01.167765 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4142f7-adff-47cd-8c8a-b86cd735b8f9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:22:01.169680 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4142f7-adff-47cd-8c8a-b86cd735b8f9 admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:22:01.277884 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1329/2678] 10.222.0.46 () {60 vars in 1144 bytes} [Mon Oct 10 12:22:01 2022] POST /identity/v3/users => generated 287 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 10 12:22:01.282123 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced3c156-f5fb-4e97-9de6-a5411875fa0d admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:22:01.296674 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced3c156-f5fb-4e97-9de6-a5411875fa0d admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:22:01.297959 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13132843-2b71-4eac-bd2b-73e2e35a5fdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:22:01.298568 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13132843-2b71-4eac-bd2b-73e2e35a5fdb admin admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:22:01.298879 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13132843-2b71-4eac-bd2b-73e2e35a5fdb admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:22:01.299160 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13132843-2b71-4eac-bd2b-73e2e35a5fdb admin admin] PATH_INFO: `/v3/roles` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:22:01.300313 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13132843-2b71-4eac-bd2b-73e2e35a5fdb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:22:01.301948 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13132843-2b71-4eac-bd2b-73e2e35a5fdb admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:22:01.312198 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1350/2679] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 10 12:22:01 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 12:22:01.316925 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4142f7-adff-47cd-8c8a-b86cd735b8f9 admin admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:22:01.327768 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4142f7-adff-47cd-8c8a-b86cd735b8f9 admin admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:22:01.332719 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cef8a73-b202-49e3-bc57-4f9f56e672a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:22:01.333367 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cef8a73-b202-49e3-bc57-4f9f56e672a4 admin admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:22:01.333656 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cef8a73-b202-49e3-bc57-4f9f56e672a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:22:01.333936 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cef8a73-b202-49e3-bc57-4f9f56e672a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:22:01.334621 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cef8a73-b202-49e3-bc57-4f9f56e672a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:22:01.336576 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cef8a73-b202-49e3-bc57-4f9f56e672a4 admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:22:01.347103 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1330/2680] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 10 12:22:01 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 12:22:01.351542 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13132843-2b71-4eac-bd2b-73e2e35a5fdb admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:22:01.361646 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13132843-2b71-4eac-bd2b-73e2e35a5fdb admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:22:01.366961 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941e3a7d-b27d-4972-b26c-aec48ed62bd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:22:01.367643 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941e3a7d-b27d-4972-b26c-aec48ed62bd4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:22:01.367956 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941e3a7d-b27d-4972-b26c-aec48ed62bd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:22:01.368238 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941e3a7d-b27d-4972-b26c-aec48ed62bd4 admin admin] PATH_INFO: `/v3/projects/931d44e7d8c84dad9aa3c64eafd899d8/users/edddd32b0093470482df3491ed69ad6c/roles/b2cb2a6e818e47338ce071e6fedfc673` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:22:01.384483 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941e3a7d-b27d-4972-b26c-aec48ed62bd4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=931d44e7d8c84dad9aa3c64eafd899d8, user_id=edddd32b0093470482df3491ed69ad6c, role_id=b2cb2a6e818e47338ce071e6fedfc673)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:22:01.387278 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941e3a7d-b27d-4972-b26c-aec48ed62bd4 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:22:01.435486 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cef8a73-b202-49e3-bc57-4f9f56e672a4 admin admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:22:01.437222 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-6cef8a73-b202-49e3-bc57-4f9f56e672a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:22:01.502746 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cef8a73-b202-49e3-bc57-4f9f56e672a4 admin admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:22:01.506379 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595f8662-fe8d-4ec0-8118-cc1aa0074515 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3ae093470484b93b23dcfae1657df9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad3772f7b6a47c28b7ba37a9316a3c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:22:01.507033 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595f8662-fe8d-4ec0-8118-cc1aa0074515 service cinder] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:22:01.507310 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595f8662-fe8d-4ec0-8118-cc1aa0074515 service cinder] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:22:01.507580 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595f8662-fe8d-4ec0-8118-cc1aa0074515 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:22:01.509357 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-595f8662-fe8d-4ec0-8118-cc1aa0074515 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:22:01.523188 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1351/2681] 10.222.0.46 () {60 vars in 1483 bytes} [Mon Oct 10 12:22:01 2022] PUT /identity/v3/projects/931d44e7d8c84dad9aa3c64eafd899d8/users/edddd32b0093470482df3491ed69ad6c/roles/b2cb2a6e818e47338ce071e6fedfc673 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:22:01.533569 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533fa225-976d-46ea-8d8b-c6fd3a778cf8 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:22:01.533963 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533fa225-976d-46ea-8d8b-c6fd3a778cf8 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:22:01.534929 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533fa225-976d-46ea-8d8b-c6fd3a778cf8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:22:01.566128 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-533fa225-976d-46ea-8d8b-c6fd3a778cf8 None None] MFA Rules not processed for user `edddd32b0093470482df3491ed69ad6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:22:01.567426 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-533fa225-976d-46ea-8d8b-c6fd3a778cf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:22:01.583385 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595f8662-fe8d-4ec0-8118-cc1aa0074515 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:22:01.584955 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595f8662-fe8d-4ec0-8118-cc1aa0074515 service cinder] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:22:01.596437 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1331/2682] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 10 12:22:01 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:22:01.644150 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1352/2683] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:22:01 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 10 12:22:01.655143 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595f8662-fe8d-4ec0-8118-cc1aa0074515 service cinder] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:22:01.661443 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-595f8662-fe8d-4ec0-8118-cc1aa0074515 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:22:01.669127 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595f8662-fe8d-4ec0-8118-cc1aa0074515 service cinder] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:22:01.724335 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c88086e-2a40-452c-92d6-0775982daf4e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3ae093470484b93b23dcfae1657df9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad3772f7b6a47c28b7ba37a9316a3c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:22:01.725024 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c88086e-2a40-452c-92d6-0775982daf4e service cinder] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:22:01.725310 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c88086e-2a40-452c-92d6-0775982daf4e service cinder] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:22:01.725585 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c88086e-2a40-452c-92d6-0775982daf4e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:22:01.734392 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c88086e-2a40-452c-92d6-0775982daf4e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:22:01.735858 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c88086e-2a40-452c-92d6-0775982daf4e service cinder] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:22:01.745470 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1332/2684] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 10 12:22:01 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:22:03.492210 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99280a61-8aca-4aac-bc80-839c2ca5ca81 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:22:03.494109 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99280a61-8aca-4aac-bc80-839c2ca5ca81 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:22:03.494109 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99280a61-8aca-4aac-bc80-839c2ca5ca81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:22:03.501229 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:22:03.512456 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-99280a61-8aca-4aac-bc80-839c2ca5ca81 None None] MFA Rules not processed for user `edddd32b0093470482df3491ed69ad6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:22:03.513649 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-99280a61-8aca-4aac-bc80-839c2ca5ca81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:22:03.521120 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1353/2685] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:22:03 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 10 12:22:05.747166 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c88086e-2a40-452c-92d6-0775982daf4e service cinder] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:22:05.748673 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-7c88086e-2a40-452c-92d6-0775982daf4e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:22:05.754339 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:22:05.818234 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c88086e-2a40-452c-92d6-0775982daf4e service cinder] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:22:05.823494 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556fd735-edca-45b2-984a-1600799db830 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:22:05.824134 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556fd735-edca-45b2-984a-1600799db830 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:22:05.824414 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556fd735-edca-45b2-984a-1600799db830 admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:22:05.824693 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556fd735-edca-45b2-984a-1600799db830 admin admin] PATH_INFO: `/v3/users/af6a0f97f1a2481ea3a573ecb2560e90` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:22:05.825906 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556fd735-edca-45b2-984a-1600799db830 admin admin] RBAC: Authorizing `identity:delete_user(user_id=af6a0f97f1a2481ea3a573ecb2560e90)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:22:05.828016 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556fd735-edca-45b2-984a-1600799db830 admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:22:06.038660 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-556fd735-edca-45b2-984a-1600799db830 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'af6a0f97f1a2481ea3a573ecb2560e90'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:22:06.045539 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-556fd735-edca-45b2-984a-1600799db830 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'af6a0f97f1a2481ea3a573ecb2560e90'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:22:06.046222 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-556fd735-edca-45b2-984a-1600799db830 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'af6a0f97f1a2481ea3a573ecb2560e90'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:22:06.151298 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-556fd735-edca-45b2-984a-1600799db830 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'af6a0f97f1a2481ea3a573ecb2560e90'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:22:06.165331 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-556fd735-edca-45b2-984a-1600799db830 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'af6a0f97f1a2481ea3a573ecb2560e90'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:22:06.171572 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1333/2686] 10.222.0.46 () {58 vars in 1224 bytes} [Mon Oct 10 12:22:05 2022] DELETE /identity/v3/users/af6a0f97f1a2481ea3a573ecb2560e90 => generated 0 bytes in 425 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:22:06.176410 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374a5f1e-ccfb-4d44-a266-1b60f0a60c8a None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:22:06.176410 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374a5f1e-ccfb-4d44-a266-1b60f0a60c8a None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:22:06.176410 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374a5f1e-ccfb-4d44-a266-1b60f0a60c8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:22:06.199458 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-374a5f1e-ccfb-4d44-a266-1b60f0a60c8a None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:22:06.200646 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-374a5f1e-ccfb-4d44-a266-1b60f0a60c8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:22:06.260088 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1354/2687] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:22:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:22:06.270113 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556fd735-edca-45b2-984a-1600799db830 admin admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:22:06.271557 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-556fd735-edca-45b2-984a-1600799db830 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:22:06.337979 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556fd735-edca-45b2-984a-1600799db830 admin admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:22:06.342130 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55090fa2-a02b-43c1-a99a-f553c6dbf888 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58b1c7c85302421cba823ace4201861f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad3772f7b6a47c28b7ba37a9316a3c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:22:06.342778 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55090fa2-a02b-43c1-a99a-f553c6dbf888 service neutron] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:22:06.348865 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55090fa2-a02b-43c1-a99a-f553c6dbf888 service neutron] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:22:06.348865 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55090fa2-a02b-43c1-a99a-f553c6dbf888 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:22:06.359264 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55090fa2-a02b-43c1-a99a-f553c6dbf888 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:22:06.360858 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55090fa2-a02b-43c1-a99a-f553c6dbf888 service neutron] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:22:06.372066 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1334/2688] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 10 12:22:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:22:07.074816 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374a5f1e-ccfb-4d44-a266-1b60f0a60c8a None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:22:07.076265 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-374a5f1e-ccfb-4d44-a266-1b60f0a60c8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:22:07.096656 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374a5f1e-ccfb-4d44-a266-1b60f0a60c8a None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:22:07.096656 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db385ff-bd10-4092-afe9-1bb7d1406f1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:22:07.096656 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db385ff-bd10-4092-afe9-1bb7d1406f1b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:22:07.096656 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db385ff-bd10-4092-afe9-1bb7d1406f1b admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:22:07.097040 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db385ff-bd10-4092-afe9-1bb7d1406f1b admin admin] PATH_INFO: `/v3/projects/db0440d186134397809e219a27deff13` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:22:07.098090 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db385ff-bd10-4092-afe9-1bb7d1406f1b admin admin] RBAC: Authorizing `identity:delete_project(project_id=db0440d186134397809e219a27deff13)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:22:07.100569 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db385ff-bd10-4092-afe9-1bb7d1406f1b admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:22:07.259488 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-0db385ff-bd10-4092-afe9-1bb7d1406f1b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'db0440d186134397809e219a27deff13', 'request_id': 'req-0db385ff-bd10-4092-afe9-1bb7d1406f1b'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:22:07.265345 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-0db385ff-bd10-4092-afe9-1bb7d1406f1b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'db0440d186134397809e219a27deff13', 'request_id': 'req-0db385ff-bd10-4092-afe9-1bb7d1406f1b'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:22:07.272227 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-0db385ff-bd10-4092-afe9-1bb7d1406f1b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'db0440d186134397809e219a27deff13', 'request_id': 'req-0db385ff-bd10-4092-afe9-1bb7d1406f1b'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:22:07.380060 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-0db385ff-bd10-4092-afe9-1bb7d1406f1b admin admin] The token cache is being invalidate because project db0440d186134397809e219a27deff13 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81677) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 10 12:22:07.380470 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-0db385ff-bd10-4092-afe9-1bb7d1406f1b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:22:07.381878 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-0db385ff-bd10-4092-afe9-1bb7d1406f1b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:22:07.383742 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1355/2689] 10.222.0.46 () {58 vars in 1233 bytes} [Mon Oct 10 12:22:07 2022] DELETE /identity/v3/projects/db0440d186134397809e219a27deff13 => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 10 12:22:07.390873 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55090fa2-a02b-43c1-a99a-f553c6dbf888 service neutron] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:22:07.402083 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55090fa2-a02b-43c1-a99a-f553c6dbf888 service neutron] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:22:07.433396 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:22:07.483660 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f8e931-3fc2-424a-8831-4f08604e269e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:22:07.484664 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f8e931-3fc2-424a-8831-4f08604e269e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:22:07.485926 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f8e931-3fc2-424a-8831-4f08604e269e admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:22:07.486348 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f8e931-3fc2-424a-8831-4f08604e269e admin admin] PATH_INFO: `/v3/users/6fdc3a6422c04ddea95915a8de497c2c` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:22:07.487661 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f8e931-3fc2-424a-8831-4f08604e269e admin admin] RBAC: Authorizing `identity:delete_user(user_id=6fdc3a6422c04ddea95915a8de497c2c)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:22:07.490261 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f8e931-3fc2-424a-8831-4f08604e269e admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:22:07.910462 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-06f8e931-3fc2-424a-8831-4f08604e269e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6fdc3a6422c04ddea95915a8de497c2c'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:22:07.922092 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-06f8e931-3fc2-424a-8831-4f08604e269e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6fdc3a6422c04ddea95915a8de497c2c'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:22:07.922875 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-06f8e931-3fc2-424a-8831-4f08604e269e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6fdc3a6422c04ddea95915a8de497c2c'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:22:08.013370 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-06f8e931-3fc2-424a-8831-4f08604e269e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6fdc3a6422c04ddea95915a8de497c2c'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:22:08.032763 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-06f8e931-3fc2-424a-8831-4f08604e269e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6fdc3a6422c04ddea95915a8de497c2c'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:22:08.039710 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1335/2690] 10.222.0.46 () {58 vars in 1224 bytes} [Mon Oct 10 12:22:07 2022] DELETE /identity/v3/users/6fdc3a6422c04ddea95915a8de497c2c => generated 0 bytes in 652 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 10 12:22:08.508192 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db385ff-bd10-4092-afe9-1bb7d1406f1b admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:22:08.509850 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-0db385ff-bd10-4092-afe9-1bb7d1406f1b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:22:08.540233 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db385ff-bd10-4092-afe9-1bb7d1406f1b admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:22:08.606638 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa62e701-797f-41de-8ab1-0cf2acc0f89e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:22:08.607613 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa62e701-797f-41de-8ab1-0cf2acc0f89e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:22:08.607896 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa62e701-797f-41de-8ab1-0cf2acc0f89e admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:22:08.608171 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa62e701-797f-41de-8ab1-0cf2acc0f89e admin admin] PATH_INFO: `/v3/projects/808a9a8554544780957e3d75f523de48` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:22:08.609330 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa62e701-797f-41de-8ab1-0cf2acc0f89e admin admin] RBAC: Authorizing `identity:delete_project(project_id=808a9a8554544780957e3d75f523de48)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:22:08.611416 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa62e701-797f-41de-8ab1-0cf2acc0f89e admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:22:08.640788 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:22:08.785757 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-fa62e701-797f-41de-8ab1-0cf2acc0f89e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '808a9a8554544780957e3d75f523de48', 'request_id': 'req-fa62e701-797f-41de-8ab1-0cf2acc0f89e'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:22:08.791771 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-fa62e701-797f-41de-8ab1-0cf2acc0f89e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '808a9a8554544780957e3d75f523de48', 'request_id': 'req-fa62e701-797f-41de-8ab1-0cf2acc0f89e'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:22:08.798182 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-fa62e701-797f-41de-8ab1-0cf2acc0f89e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '808a9a8554544780957e3d75f523de48', 'request_id': 'req-fa62e701-797f-41de-8ab1-0cf2acc0f89e'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:22:08.909836 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-fa62e701-797f-41de-8ab1-0cf2acc0f89e admin admin] The token cache is being invalidate because project 808a9a8554544780957e3d75f523de48 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81677) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 10 12:22:08.910130 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-fa62e701-797f-41de-8ab1-0cf2acc0f89e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:22:08.910808 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-fa62e701-797f-41de-8ab1-0cf2acc0f89e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:22:08.912389 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1356/2691] 10.222.0.46 () {58 vars in 1233 bytes} [Mon Oct 10 12:22:08 2022] DELETE /identity/v3/projects/808a9a8554544780957e3d75f523de48 => generated 0 bytes in 405 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:22:08.916455 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f8e931-3fc2-424a-8831-4f08604e269e admin admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:22:08.917966 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-06f8e931-3fc2-424a-8831-4f08604e269e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:22:08.931783 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f8e931-3fc2-424a-8831-4f08604e269e admin admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:22:08.995659 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70cb36f-244c-4aab-8f65-0da1e66918e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:22:08.996599 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70cb36f-244c-4aab-8f65-0da1e66918e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:22:08.996877 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70cb36f-244c-4aab-8f65-0da1e66918e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:22:08.997159 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70cb36f-244c-4aab-8f65-0da1e66918e3 admin admin] PATH_INFO: `/v3/users/912b41f3342d4035b552f8e0b5311f48` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:22:08.998520 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70cb36f-244c-4aab-8f65-0da1e66918e3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=912b41f3342d4035b552f8e0b5311f48)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:22:09.000564 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70cb36f-244c-4aab-8f65-0da1e66918e3 admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:22:09.219092 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=82,keystone:DELETE=17,keystone:INSERT=1 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:22:09.220891 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-a70cb36f-244c-4aab-8f65-0da1e66918e3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '912b41f3342d4035b552f8e0b5311f48'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:22:09.229882 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-a70cb36f-244c-4aab-8f65-0da1e66918e3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '912b41f3342d4035b552f8e0b5311f48'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:22:09.230602 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-a70cb36f-244c-4aab-8f65-0da1e66918e3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '912b41f3342d4035b552f8e0b5311f48'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:22:09.290319 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-a70cb36f-244c-4aab-8f65-0da1e66918e3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '912b41f3342d4035b552f8e0b5311f48'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:22:09.305853 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-a70cb36f-244c-4aab-8f65-0da1e66918e3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '912b41f3342d4035b552f8e0b5311f48'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:22:09.310876 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1336/2692] 10.222.0.46 () {58 vars in 1224 bytes} [Mon Oct 10 12:22:08 2022] DELETE /identity/v3/users/912b41f3342d4035b552f8e0b5311f48 => generated 0 bytes in 395 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:22:09.766410 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa62e701-797f-41de-8ab1-0cf2acc0f89e admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:22:09.767919 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-fa62e701-797f-41de-8ab1-0cf2acc0f89e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:22:09.783612 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa62e701-797f-41de-8ab1-0cf2acc0f89e admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:22:09.849060 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a134b1-4026-410a-ad16-704a72b54b91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:22:09.849630 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a134b1-4026-410a-ad16-704a72b54b91 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:22:09.849914 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a134b1-4026-410a-ad16-704a72b54b91 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:22:09.850264 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a134b1-4026-410a-ad16-704a72b54b91 admin admin] PATH_INFO: `/v3/projects/0c75e6565ae6473c96760c4dd7b22e4c` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:22:09.852189 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a134b1-4026-410a-ad16-704a72b54b91 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0c75e6565ae6473c96760c4dd7b22e4c)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:22:09.854190 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a134b1-4026-410a-ad16-704a72b54b91 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:22:09.990458 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-88a134b1-4026-410a-ad16-704a72b54b91 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0c75e6565ae6473c96760c4dd7b22e4c', 'request_id': 'req-88a134b1-4026-410a-ad16-704a72b54b91'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:22:09.996044 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-88a134b1-4026-410a-ad16-704a72b54b91 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0c75e6565ae6473c96760c4dd7b22e4c', 'request_id': 'req-88a134b1-4026-410a-ad16-704a72b54b91'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:22:10.002252 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-88a134b1-4026-410a-ad16-704a72b54b91 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0c75e6565ae6473c96760c4dd7b22e4c', 'request_id': 'req-88a134b1-4026-410a-ad16-704a72b54b91'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:22:10.075903 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-88a134b1-4026-410a-ad16-704a72b54b91 admin admin] The token cache is being invalidate because project 0c75e6565ae6473c96760c4dd7b22e4c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81677) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 10 12:22:10.079660 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-88a134b1-4026-410a-ad16-704a72b54b91 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:22:10.079660 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-88a134b1-4026-410a-ad16-704a72b54b91 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:22:10.079660 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1357/2693] 10.222.0.46 () {58 vars in 1233 bytes} [Mon Oct 10 12:22:09 2022] DELETE /identity/v3/projects/0c75e6565ae6473c96760c4dd7b22e4c => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:22:10.086607 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6349617d-1aa7-4ae2-8492-80fb4b404eec None None] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:22:10.087111 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6349617d-1aa7-4ae2-8492-80fb4b404eec None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:22:10.087211 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6349617d-1aa7-4ae2-8492-80fb4b404eec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:22:10.108375 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.auth.core [None req-6349617d-1aa7-4ae2-8492-80fb4b404eec None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:22:10.109527 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-6349617d-1aa7-4ae2-8492-80fb4b404eec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:22:10.171059 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1337/2694] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:22:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:22:10.179444 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef58f7d9-ffbf-423e-a78f-1e196149ca78 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:22:10.179724 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef58f7d9-ffbf-423e-a78f-1e196149ca78 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:22:10.180001 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef58f7d9-ffbf-423e-a78f-1e196149ca78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:22:10.199491 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-ef58f7d9-ffbf-423e-a78f-1e196149ca78 None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:22:10.200677 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-ef58f7d9-ffbf-423e-a78f-1e196149ca78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:22:10.208622 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1358/2695] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:22:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:22:10.213782 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6349617d-1aa7-4ae2-8492-80fb4b404eec None None] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:22:10.223894 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6349617d-1aa7-4ae2-8492-80fb4b404eec None None] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:22:10.229143 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8881e9a2-3b96-4d6f-b92a-d49fc99c00b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:22:10.229741 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8881e9a2-3b96-4d6f-b92a-d49fc99c00b2 admin admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:22:10.230022 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8881e9a2-3b96-4d6f-b92a-d49fc99c00b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:22:10.230300 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8881e9a2-3b96-4d6f-b92a-d49fc99c00b2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:22:10.230995 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8881e9a2-3b96-4d6f-b92a-d49fc99c00b2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 12:22:10.231300 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8881e9a2-3b96-4d6f-b92a-d49fc99c00b2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:22:10.232896 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8881e9a2-3b96-4d6f-b92a-d49fc99c00b2 admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:22:10.244938 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1338/2696] 10.222.0.46 () {58 vars in 1153 bytes} [Mon Oct 10 12:22:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 12:22:10.249337 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef58f7d9-ffbf-423e-a78f-1e196149ca78 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:22:10.259216 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef58f7d9-ffbf-423e-a78f-1e196149ca78 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:22:10.264035 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce3b7d6-d893-40ed-b13b-ab6dc82a20d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:22:10.264634 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce3b7d6-d893-40ed-b13b-ab6dc82a20d1 admin admin] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:22:10.264923 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce3b7d6-d893-40ed-b13b-ab6dc82a20d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:22:10.265200 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce3b7d6-d893-40ed-b13b-ab6dc82a20d1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:22:10.265846 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce3b7d6-d893-40ed-b13b-ab6dc82a20d1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:22:10.267898 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce3b7d6-d893-40ed-b13b-ab6dc82a20d1 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:22:10.365604 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1359/2697] 10.222.0.46 () {60 vars in 1153 bytes} [Mon Oct 10 12:22:10 2022] POST /identity/v3/projects => generated 393 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 12:22:10.369728 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8881e9a2-3b96-4d6f-b92a-d49fc99c00b2 admin admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:22:10.379635 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8881e9a2-3b96-4d6f-b92a-d49fc99c00b2 admin admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:22:10.434308 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d6cf12-ef59-41ea-ab22-430d7d5cd2ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:22:10.434928 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d6cf12-ef59-41ea-ab22-430d7d5cd2ea admin admin] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:22:10.435206 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d6cf12-ef59-41ea-ab22-430d7d5cd2ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:22:10.435478 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d6cf12-ef59-41ea-ab22-430d7d5cd2ea admin admin] PATH_INFO: `/v3/users` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:22:10.436116 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d6cf12-ef59-41ea-ab22-430d7d5cd2ea admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:22:10.437994 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d6cf12-ef59-41ea-ab22-430d7d5cd2ea admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:22:10.549935 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1339/2698] 10.222.0.46 () {60 vars in 1144 bytes} [Mon Oct 10 12:22:10 2022] POST /identity/v3/users => generated 297 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 10 12:22:10.554133 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce3b7d6-d893-40ed-b13b-ab6dc82a20d1 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:22:10.571899 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce3b7d6-d893-40ed-b13b-ab6dc82a20d1 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:22:10.576721 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74233c40-2e2d-4358-8205-f0ad7be0b38c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:22:10.577306 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74233c40-2e2d-4358-8205-f0ad7be0b38c admin admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:22:10.577584 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74233c40-2e2d-4358-8205-f0ad7be0b38c admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:22:10.577855 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74233c40-2e2d-4358-8205-f0ad7be0b38c admin admin] PATH_INFO: `/v3/roles` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:22:10.578502 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74233c40-2e2d-4358-8205-f0ad7be0b38c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:22:10.580143 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74233c40-2e2d-4358-8205-f0ad7be0b38c admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:22:10.591909 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1360/2699] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 10 12:22:10 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 12:22:10.596215 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d6cf12-ef59-41ea-ab22-430d7d5cd2ea admin admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:22:10.606461 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d6cf12-ef59-41ea-ab22-430d7d5cd2ea admin admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:22:10.611180 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7826bdb-af8f-4681-88dc-248241c7c381 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:22:10.611779 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7826bdb-af8f-4681-88dc-248241c7c381 admin admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:22:10.612064 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7826bdb-af8f-4681-88dc-248241c7c381 admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:22:10.612336 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7826bdb-af8f-4681-88dc-248241c7c381 admin admin] PATH_INFO: `/v3/roles` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:22:10.612983 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7826bdb-af8f-4681-88dc-248241c7c381 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:22:10.614571 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7826bdb-af8f-4681-88dc-248241c7c381 admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:22:10.625784 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1340/2700] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 10 12:22:10 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 12:22:10.632911 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74233c40-2e2d-4358-8205-f0ad7be0b38c admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:22:10.638172 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=84,keystone:DELETE=13,keystone:INSERT=3 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:22:10.643565 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74233c40-2e2d-4358-8205-f0ad7be0b38c admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:22:10.648820 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656c035a-3da2-44aa-93f1-356108f4b239 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:22:10.649474 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656c035a-3da2-44aa-93f1-356108f4b239 admin admin] REQUEST_METHOD: `PUT` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:22:10.649774 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656c035a-3da2-44aa-93f1-356108f4b239 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:22:10.650047 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656c035a-3da2-44aa-93f1-356108f4b239 admin admin] PATH_INFO: `/v3/projects/8ef95c3116be40ebadade4cb2175948e/users/09cf8764085d4a95bc4d515991ac2a2a/roles/b2cb2a6e818e47338ce071e6fedfc673` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:22:10.664705 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656c035a-3da2-44aa-93f1-356108f4b239 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8ef95c3116be40ebadade4cb2175948e, user_id=09cf8764085d4a95bc4d515991ac2a2a, role_id=b2cb2a6e818e47338ce071e6fedfc673)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:22:10.667391 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656c035a-3da2-44aa-93f1-356108f4b239 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:22:10.749176 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1361/2701] 10.222.0.46 () {60 vars in 1483 bytes} [Mon Oct 10 12:22:10 2022] PUT /identity/v3/projects/8ef95c3116be40ebadade4cb2175948e/users/09cf8764085d4a95bc4d515991ac2a2a/roles/b2cb2a6e818e47338ce071e6fedfc673 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:22:10.759974 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde49d9f-e25f-4c40-aa55-31e537e59cf0 None None] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:22:10.760262 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde49d9f-e25f-4c40-aa55-31e537e59cf0 None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:22:10.760541 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde49d9f-e25f-4c40-aa55-31e537e59cf0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:22:10.791089 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.auth.core [None req-bde49d9f-e25f-4c40-aa55-31e537e59cf0 None None] MFA Rules not processed for user `09cf8764085d4a95bc4d515991ac2a2a`. Rule list: `[]` (Enabled: `True`). {{(pid=81676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:22:10.792298 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-bde49d9f-e25f-4c40-aa55-31e537e59cf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:22:10.809187 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:22:10.857120 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1341/2702] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:22:10 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:22:10.862420 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656c035a-3da2-44aa-93f1-356108f4b239 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:22:10.925171 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656c035a-3da2-44aa-93f1-356108f4b239 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:22:10.930529 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c0065f-b0ad-4ea2-adb2-1848568493a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:22:10.931188 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c0065f-b0ad-4ea2-adb2-1848568493a0 admin admin] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:22:10.931470 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c0065f-b0ad-4ea2-adb2-1848568493a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:22:10.931776 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c0065f-b0ad-4ea2-adb2-1848568493a0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:22:10.932403 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c0065f-b0ad-4ea2-adb2-1848568493a0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:22:10.934307 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c0065f-b0ad-4ea2-adb2-1848568493a0 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:22:11.002676 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1362/2703] 10.222.0.46 () {60 vars in 1153 bytes} [Mon Oct 10 12:22:10 2022] POST /identity/v3/projects => generated 391 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 12:22:11.007766 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde49d9f-e25f-4c40-aa55-31e537e59cf0 None None] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:22:11.019023 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde49d9f-e25f-4c40-aa55-31e537e59cf0 None None] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:22:11.078080 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83eaa0a7-809e-400c-89d8-473bfabc3b17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:22:11.078721 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83eaa0a7-809e-400c-89d8-473bfabc3b17 admin admin] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:22:11.079012 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83eaa0a7-809e-400c-89d8-473bfabc3b17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:22:11.079325 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83eaa0a7-809e-400c-89d8-473bfabc3b17 admin admin] PATH_INFO: `/v3/users` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:22:11.079966 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83eaa0a7-809e-400c-89d8-473bfabc3b17 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:22:11.081940 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83eaa0a7-809e-400c-89d8-473bfabc3b17 admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:22:11.153357 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1342/2704] 10.222.0.46 () {60 vars in 1144 bytes} [Mon Oct 10 12:22:11 2022] POST /identity/v3/users => generated 302 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 12:22:11.157275 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c0065f-b0ad-4ea2-adb2-1848568493a0 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:22:11.167946 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c0065f-b0ad-4ea2-adb2-1848568493a0 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:22:11.172981 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9b5522-d6d7-4d96-9171-bfa2aeffcccb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:22:11.173620 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9b5522-d6d7-4d96-9171-bfa2aeffcccb admin admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:22:11.173928 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9b5522-d6d7-4d96-9171-bfa2aeffcccb admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:22:11.174201 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9b5522-d6d7-4d96-9171-bfa2aeffcccb admin admin] PATH_INFO: `/v3/roles` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:22:11.175698 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9b5522-d6d7-4d96-9171-bfa2aeffcccb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:22:11.176537 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9b5522-d6d7-4d96-9171-bfa2aeffcccb admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:22:11.187974 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1363/2705] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 10 12:22:11 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 12:22:11.192449 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83eaa0a7-809e-400c-89d8-473bfabc3b17 admin admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:22:11.203454 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83eaa0a7-809e-400c-89d8-473bfabc3b17 admin admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:22:11.208263 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adeba784-95b3-4e6e-8115-7a2d15d45a53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:22:11.208917 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adeba784-95b3-4e6e-8115-7a2d15d45a53 admin admin] REQUEST_METHOD: `PUT` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:22:11.209201 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adeba784-95b3-4e6e-8115-7a2d15d45a53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:22:11.209480 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adeba784-95b3-4e6e-8115-7a2d15d45a53 admin admin] PATH_INFO: `/v3/projects/9d26079a048f44ffab880a063bcec589/users/41bd1e7b92dd468ab948e52f1ec3c406/roles/0d822a0674b6495fac623877a3b38025` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:22:11.224751 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adeba784-95b3-4e6e-8115-7a2d15d45a53 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9d26079a048f44ffab880a063bcec589, user_id=41bd1e7b92dd468ab948e52f1ec3c406, role_id=0d822a0674b6495fac623877a3b38025)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:22:11.228081 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adeba784-95b3-4e6e-8115-7a2d15d45a53 admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:22:11.325239 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1343/2706] 10.222.0.46 () {60 vars in 1483 bytes} [Mon Oct 10 12:22:11 2022] PUT /identity/v3/projects/9d26079a048f44ffab880a063bcec589/users/41bd1e7b92dd468ab948e52f1ec3c406/roles/0d822a0674b6495fac623877a3b38025 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 10 12:22:11.329258 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9b5522-d6d7-4d96-9171-bfa2aeffcccb admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:22:11.339691 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9b5522-d6d7-4d96-9171-bfa2aeffcccb admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:22:11.395651 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8927c81-6392-4bd9-83a6-6b88b325087c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:22:11.396286 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8927c81-6392-4bd9-83a6-6b88b325087c admin admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:22:11.396566 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8927c81-6392-4bd9-83a6-6b88b325087c admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:22:11.396840 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8927c81-6392-4bd9-83a6-6b88b325087c admin admin] PATH_INFO: `/v3/roles` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:22:11.397979 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8927c81-6392-4bd9-83a6-6b88b325087c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:22:11.399592 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8927c81-6392-4bd9-83a6-6b88b325087c admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:22:11.409632 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1364/2707] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 10 12:22:11 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 10 12:22:11.414348 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adeba784-95b3-4e6e-8115-7a2d15d45a53 admin admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:22:11.424578 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adeba784-95b3-4e6e-8115-7a2d15d45a53 admin admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:22:11.429209 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690e99ac-8fc4-4c18-bcb6-d1495feaef2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:22:11.429845 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690e99ac-8fc4-4c18-bcb6-d1495feaef2b admin admin] REQUEST_METHOD: `PUT` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:22:11.430124 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690e99ac-8fc4-4c18-bcb6-d1495feaef2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:22:11.430396 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690e99ac-8fc4-4c18-bcb6-d1495feaef2b admin admin] PATH_INFO: `/v3/projects/9d26079a048f44ffab880a063bcec589/users/41bd1e7b92dd468ab948e52f1ec3c406/roles/0d822a0674b6495fac623877a3b38025` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:22:11.432339 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690e99ac-8fc4-4c18-bcb6-d1495feaef2b admin admin] RBAC: Authorizing `identity:create_grant(project_id=9d26079a048f44ffab880a063bcec589, user_id=41bd1e7b92dd468ab948e52f1ec3c406, role_id=0d822a0674b6495fac623877a3b38025)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:22:11.435002 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690e99ac-8fc4-4c18-bcb6-d1495feaef2b admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:22:11.445153 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1344/2708] 10.222.0.46 () {60 vars in 1483 bytes} [Mon Oct 10 12:22:11 2022] PUT /identity/v3/projects/9d26079a048f44ffab880a063bcec589/users/41bd1e7b92dd468ab948e52f1ec3c406/roles/0d822a0674b6495fac623877a3b38025 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:22:11.456348 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed9e5f7-9712-4a37-87d3-73bf5b175492 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:22:11.456635 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed9e5f7-9712-4a37-87d3-73bf5b175492 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:22:11.456913 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed9e5f7-9712-4a37-87d3-73bf5b175492 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:22:11.487509 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-bed9e5f7-9712-4a37-87d3-73bf5b175492 None None] MFA Rules not processed for user `41bd1e7b92dd468ab948e52f1ec3c406`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:22:11.488722 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-bed9e5f7-9712-4a37-87d3-73bf5b175492 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:22:11.526106 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:22:11.555243 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1365/2709] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:22:11 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:22:11.566071 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690e99ac-8fc4-4c18-bcb6-d1495feaef2b admin admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:22:11.567785 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-690e99ac-8fc4-4c18-bcb6-d1495feaef2b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:22:11.627326 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690e99ac-8fc4-4c18-bcb6-d1495feaef2b admin admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:22:11.630995 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001ce870-4e8c-41b1-a53d-319afb0afc4b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3ae093470484b93b23dcfae1657df9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad3772f7b6a47c28b7ba37a9316a3c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:22:11.631603 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001ce870-4e8c-41b1-a53d-319afb0afc4b service cinder] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:22:11.631909 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001ce870-4e8c-41b1-a53d-319afb0afc4b service cinder] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:22:11.632186 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001ce870-4e8c-41b1-a53d-319afb0afc4b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:22:11.641948 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001ce870-4e8c-41b1-a53d-319afb0afc4b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:22:11.643412 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001ce870-4e8c-41b1-a53d-319afb0afc4b service cinder] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:22:11.653703 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1345/2710] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 10 12:22:11 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:22:11.700654 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed9e5f7-9712-4a37-87d3-73bf5b175492 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:22:11.702068 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-bed9e5f7-9712-4a37-87d3-73bf5b175492 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:22:11.714642 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed9e5f7-9712-4a37-87d3-73bf5b175492 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:22:11.718249 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cb35fd-6fd5-442a-8dd1-be9a1038d3cf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3ae093470484b93b23dcfae1657df9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad3772f7b6a47c28b7ba37a9316a3c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:22:11.718884 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cb35fd-6fd5-442a-8dd1-be9a1038d3cf service cinder] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:22:11.719164 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cb35fd-6fd5-442a-8dd1-be9a1038d3cf service cinder] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:22:11.719434 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cb35fd-6fd5-442a-8dd1-be9a1038d3cf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:22:11.778130 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37cb35fd-6fd5-442a-8dd1-be9a1038d3cf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:22:11.779606 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37cb35fd-6fd5-442a-8dd1-be9a1038d3cf service cinder] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:22:11.788902 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1366/2711] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 10 12:22:11 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:22:11.804167 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001ce870-4e8c-41b1-a53d-319afb0afc4b service cinder] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:22:11.805587 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-001ce870-4e8c-41b1-a53d-319afb0afc4b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:22:11.820508 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:22:11.878953 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001ce870-4e8c-41b1-a53d-319afb0afc4b service cinder] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:22:11.884361 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa32e3b-a724-4b06-a767-3edaf644d616 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:22:11.885004 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa32e3b-a724-4b06-a767-3edaf644d616 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:22:11.885286 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa32e3b-a724-4b06-a767-3edaf644d616 admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:22:11.885562 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa32e3b-a724-4b06-a767-3edaf644d616 admin admin] PATH_INFO: `/v3/users/bbdf4d1c31fa4d3fa46a1e19fd422058` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:22:11.886781 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa32e3b-a724-4b06-a767-3edaf644d616 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bbdf4d1c31fa4d3fa46a1e19fd422058)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:22:11.888747 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa32e3b-a724-4b06-a767-3edaf644d616 admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:22:12.081131 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-3fa32e3b-a724-4b06-a767-3edaf644d616 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bbdf4d1c31fa4d3fa46a1e19fd422058'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:22:12.088901 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-3fa32e3b-a724-4b06-a767-3edaf644d616 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bbdf4d1c31fa4d3fa46a1e19fd422058'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:22:12.089592 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-3fa32e3b-a724-4b06-a767-3edaf644d616 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bbdf4d1c31fa4d3fa46a1e19fd422058'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:22:12.147043 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-3fa32e3b-a724-4b06-a767-3edaf644d616 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bbdf4d1c31fa4d3fa46a1e19fd422058'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:22:12.161767 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-3fa32e3b-a724-4b06-a767-3edaf644d616 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bbdf4d1c31fa4d3fa46a1e19fd422058'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:22:12.169825 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1346/2712] 10.222.0.46 () {58 vars in 1224 bytes} [Mon Oct 10 12:22:11 2022] DELETE /identity/v3/users/bbdf4d1c31fa4d3fa46a1e19fd422058 => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:22:12.174498 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b9774d-2652-45a8-ac6c-dbd4b545352c None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:22:12.175591 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b9774d-2652-45a8-ac6c-dbd4b545352c None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:22:12.176064 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b9774d-2652-45a8-ac6c-dbd4b545352c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:22:12.197353 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-b6b9774d-2652-45a8-ac6c-dbd4b545352c None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:22:12.198545 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-b6b9774d-2652-45a8-ac6c-dbd4b545352c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:22:12.262838 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1367/2713] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:22:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 10 12:22:12.272790 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa32e3b-a724-4b06-a767-3edaf644d616 admin admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:22:12.278141 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-3fa32e3b-a724-4b06-a767-3edaf644d616 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:22:12.342862 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa32e3b-a724-4b06-a767-3edaf644d616 admin admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:22:12.346423 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cff8ea-9fae-4fb9-a154-2d917ce86c17 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58b1c7c85302421cba823ace4201861f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad3772f7b6a47c28b7ba37a9316a3c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:22:12.347093 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cff8ea-9fae-4fb9-a154-2d917ce86c17 service neutron] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:22:12.347369 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cff8ea-9fae-4fb9-a154-2d917ce86c17 service neutron] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:22:12.347639 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cff8ea-9fae-4fb9-a154-2d917ce86c17 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:22:12.356781 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3cff8ea-9fae-4fb9-a154-2d917ce86c17 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:22:12.358195 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3cff8ea-9fae-4fb9-a154-2d917ce86c17 service neutron] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:22:12.368033 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1347/2714] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 10 12:22:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:22:12.871853 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b9774d-2652-45a8-ac6c-dbd4b545352c None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:22:12.873280 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-b6b9774d-2652-45a8-ac6c-dbd4b545352c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:22:12.887948 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b9774d-2652-45a8-ac6c-dbd4b545352c None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:22:12.892527 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a04477-759e-4133-9a68-21879191531f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:22:12.895478 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a04477-759e-4133-9a68-21879191531f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:22:12.895478 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a04477-759e-4133-9a68-21879191531f admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:22:12.895478 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a04477-759e-4133-9a68-21879191531f admin admin] PATH_INFO: `/v3/projects/2422eeb354644a5caa84292ee7b8b9f6` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:22:12.895917 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a04477-759e-4133-9a68-21879191531f admin admin] RBAC: Authorizing `identity:delete_project(project_id=2422eeb354644a5caa84292ee7b8b9f6)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:22:12.898138 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a04477-759e-4133-9a68-21879191531f admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:22:13.030387 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-83a04477-759e-4133-9a68-21879191531f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2422eeb354644a5caa84292ee7b8b9f6', 'request_id': 'req-83a04477-759e-4133-9a68-21879191531f'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:22:13.036150 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-83a04477-759e-4133-9a68-21879191531f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2422eeb354644a5caa84292ee7b8b9f6', 'request_id': 'req-83a04477-759e-4133-9a68-21879191531f'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:22:13.043055 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-83a04477-759e-4133-9a68-21879191531f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2422eeb354644a5caa84292ee7b8b9f6', 'request_id': 'req-83a04477-759e-4133-9a68-21879191531f'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:22:13.046617 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:22:13.120973 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-83a04477-759e-4133-9a68-21879191531f admin admin] The token cache is being invalidate because project 2422eeb354644a5caa84292ee7b8b9f6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81677) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 10 12:22:13.121180 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-83a04477-759e-4133-9a68-21879191531f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:22:13.121813 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-83a04477-759e-4133-9a68-21879191531f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:22:13.123385 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1368/2715] 10.222.0.46 () {58 vars in 1233 bytes} [Mon Oct 10 12:22:12 2022] DELETE /identity/v3/projects/2422eeb354644a5caa84292ee7b8b9f6 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:22:13.128488 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285009d5-67ba-483a-89fa-840d97f95bca None None] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:22:13.128790 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285009d5-67ba-483a-89fa-840d97f95bca None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:22:13.129071 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285009d5-67ba-483a-89fa-840d97f95bca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:22:13.150168 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.auth.core [None req-285009d5-67ba-483a-89fa-840d97f95bca None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:22:13.151460 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-285009d5-67ba-483a-89fa-840d97f95bca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:22:13.181762 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:22:13.210976 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1348/2716] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:22:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 10 12:22:13.219204 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee78fc2-b389-4832-a15b-57d23d224ff4 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:22:13.219692 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee78fc2-b389-4832-a15b-57d23d224ff4 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:22:13.219795 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee78fc2-b389-4832-a15b-57d23d224ff4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:22:13.239484 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-8ee78fc2-b389-4832-a15b-57d23d224ff4 None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:22:13.240685 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-8ee78fc2-b389-4832-a15b-57d23d224ff4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:22:13.249181 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1369/2717] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:22:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 10 12:22:13.253587 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285009d5-67ba-483a-89fa-840d97f95bca None None] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:22:13.264142 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285009d5-67ba-483a-89fa-840d97f95bca None None] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:22:13.268757 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c1478d-93ec-4472-b5a3-1edecf6b158f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:22:13.273948 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c1478d-93ec-4472-b5a3-1edecf6b158f admin admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:22:13.273948 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c1478d-93ec-4472-b5a3-1edecf6b158f admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:22:13.273948 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c1478d-93ec-4472-b5a3-1edecf6b158f admin admin] PATH_INFO: `/v3/domains` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:22:13.273948 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c1478d-93ec-4472-b5a3-1edecf6b158f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 12:22:13.273948 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c1478d-93ec-4472-b5a3-1edecf6b158f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:22:13.273948 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c1478d-93ec-4472-b5a3-1edecf6b158f admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:22:13.285126 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1349/2718] 10.222.0.46 () {58 vars in 1153 bytes} [Mon Oct 10 12:22:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 12:22:13.289855 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee78fc2-b389-4832-a15b-57d23d224ff4 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:22:13.299245 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee78fc2-b389-4832-a15b-57d23d224ff4 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:22:13.304690 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26a2783-1600-4298-8c77-3ed2a0c1b394 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:22:13.305288 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26a2783-1600-4298-8c77-3ed2a0c1b394 admin admin] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:22:13.305589 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26a2783-1600-4298-8c77-3ed2a0c1b394 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:22:13.305844 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26a2783-1600-4298-8c77-3ed2a0c1b394 admin admin] PATH_INFO: `/v3/projects` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:22:13.306471 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26a2783-1600-4298-8c77-3ed2a0c1b394 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:22:13.308473 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26a2783-1600-4298-8c77-3ed2a0c1b394 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:22:13.397545 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1370/2719] 10.222.0.46 () {60 vars in 1153 bytes} [Mon Oct 10 12:22:13 2022] POST /identity/v3/projects => generated 381 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 12:22:13.401725 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c1478d-93ec-4472-b5a3-1edecf6b158f admin admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:22:13.411976 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c1478d-93ec-4472-b5a3-1edecf6b158f admin admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:22:13.471594 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f76c405-b3f8-44db-b53f-8a9118af3a48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:22:13.472210 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f76c405-b3f8-44db-b53f-8a9118af3a48 admin admin] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:22:13.472490 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f76c405-b3f8-44db-b53f-8a9118af3a48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:22:13.472810 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f76c405-b3f8-44db-b53f-8a9118af3a48 admin admin] PATH_INFO: `/v3/users` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:22:13.473465 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f76c405-b3f8-44db-b53f-8a9118af3a48 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:22:13.475417 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f76c405-b3f8-44db-b53f-8a9118af3a48 admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:22:13.567655 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1350/2720] 10.222.0.46 () {60 vars in 1144 bytes} [Mon Oct 10 12:22:13 2022] POST /identity/v3/users => generated 291 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 12:22:13.571824 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26a2783-1600-4298-8c77-3ed2a0c1b394 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:22:13.582250 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26a2783-1600-4298-8c77-3ed2a0c1b394 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:22:13.587326 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb22988-32df-4c7f-91d1-a5838d14e6e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:22:13.587871 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb22988-32df-4c7f-91d1-a5838d14e6e0 admin admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:22:13.588148 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb22988-32df-4c7f-91d1-a5838d14e6e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:22:13.588442 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb22988-32df-4c7f-91d1-a5838d14e6e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:22:13.589058 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb22988-32df-4c7f-91d1-a5838d14e6e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:22:13.590630 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb22988-32df-4c7f-91d1-a5838d14e6e0 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:22:13.601246 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1371/2721] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 10 12:22:13 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 12:22:13.605194 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f76c405-b3f8-44db-b53f-8a9118af3a48 admin admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:22:13.615268 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f76c405-b3f8-44db-b53f-8a9118af3a48 admin admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:22:13.620813 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58fa320-412e-4d4b-a460-68c4d66654ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:22:13.621417 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58fa320-412e-4d4b-a460-68c4d66654ed admin admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:22:13.621699 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58fa320-412e-4d4b-a460-68c4d66654ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:22:13.622002 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58fa320-412e-4d4b-a460-68c4d66654ed admin admin] PATH_INFO: `/v3/roles` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:22:13.622763 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58fa320-412e-4d4b-a460-68c4d66654ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:22:13.624382 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58fa320-412e-4d4b-a460-68c4d66654ed admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:22:13.638506 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1351/2722] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 10 12:22:13 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 12:22:13.639832 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb22988-32df-4c7f-91d1-a5838d14e6e0 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:22:13.650987 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb22988-32df-4c7f-91d1-a5838d14e6e0 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:22:13.655583 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c000dadf-a430-4a91-b359-34af6ce71caa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:22:13.656263 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c000dadf-a430-4a91-b359-34af6ce71caa admin admin] REQUEST_METHOD: `PUT` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:22:13.656523 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c000dadf-a430-4a91-b359-34af6ce71caa admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:22:13.656805 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c000dadf-a430-4a91-b359-34af6ce71caa admin admin] PATH_INFO: `/v3/projects/999aa6678db8480d876992e96a9d577f/users/b4f1817a080049429541a5c5e3929cf8/roles/b2cb2a6e818e47338ce071e6fedfc673` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:22:13.677292 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c000dadf-a430-4a91-b359-34af6ce71caa admin admin] RBAC: Authorizing `identity:create_grant(project_id=999aa6678db8480d876992e96a9d577f, user_id=b4f1817a080049429541a5c5e3929cf8, role_id=b2cb2a6e818e47338ce071e6fedfc673)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:22:13.677292 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c000dadf-a430-4a91-b359-34af6ce71caa admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:22:13.757866 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1372/2723] 10.222.0.46 () {60 vars in 1483 bytes} [Mon Oct 10 12:22:13 2022] PUT /identity/v3/projects/999aa6678db8480d876992e96a9d577f/users/b4f1817a080049429541a5c5e3929cf8/roles/b2cb2a6e818e47338ce071e6fedfc673 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:22:13.778543 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9859c4-8be5-4480-85d9-46e0e7c139e0 None None] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:22:13.778942 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9859c4-8be5-4480-85d9-46e0e7c139e0 None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:22:13.779294 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9859c4-8be5-4480-85d9-46e0e7c139e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:22:13.809941 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.auth.core [None req-db9859c4-8be5-4480-85d9-46e0e7c139e0 None None] MFA Rules not processed for user `b4f1817a080049429541a5c5e3929cf8`. Rule list: `[]` (Enabled: `True`). {{(pid=81676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:22:13.811278 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-db9859c4-8be5-4480-85d9-46e0e7c139e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:22:13.879932 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1352/2724] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:22:13 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 10 12:22:13.884605 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c000dadf-a430-4a91-b359-34af6ce71caa admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:22:13.949549 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c000dadf-a430-4a91-b359-34af6ce71caa admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:22:13.954486 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663f98dc-11a8-43d4-b038-626bc3ae740e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:22:13.955238 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663f98dc-11a8-43d4-b038-626bc3ae740e admin admin] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:22:13.955524 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663f98dc-11a8-43d4-b038-626bc3ae740e admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:22:13.955795 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663f98dc-11a8-43d4-b038-626bc3ae740e admin admin] PATH_INFO: `/v3/projects` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:22:13.956432 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663f98dc-11a8-43d4-b038-626bc3ae740e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:22:13.958335 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663f98dc-11a8-43d4-b038-626bc3ae740e admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:22:14.077661 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1373/2725] 10.222.0.46 () {60 vars in 1153 bytes} [Mon Oct 10 12:22:13 2022] POST /identity/v3/projects => generated 381 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 12:22:14.081994 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9859c4-8be5-4480-85d9-46e0e7c139e0 None None] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:22:14.091093 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9859c4-8be5-4480-85d9-46e0e7c139e0 None None] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:22:14.121936 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:22:14.150368 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a363cd-8d32-4e33-b465-5d658207dd3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:22:14.151265 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a363cd-8d32-4e33-b465-5d658207dd3d admin admin] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:22:14.151747 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a363cd-8d32-4e33-b465-5d658207dd3d admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:22:14.152207 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a363cd-8d32-4e33-b465-5d658207dd3d admin admin] PATH_INFO: `/v3/users` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:22:14.153232 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a363cd-8d32-4e33-b465-5d658207dd3d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:22:14.155317 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a363cd-8d32-4e33-b465-5d658207dd3d admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:22:14.250300 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1353/2726] 10.222.0.46 () {60 vars in 1144 bytes} [Mon Oct 10 12:22:14 2022] POST /identity/v3/users => generated 297 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 12:22:14.254302 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663f98dc-11a8-43d4-b038-626bc3ae740e admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:22:14.264931 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663f98dc-11a8-43d4-b038-626bc3ae740e admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:22:14.269587 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ee1127-ecda-4698-9324-31c1cd818725 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:22:14.270179 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ee1127-ecda-4698-9324-31c1cd818725 admin admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:22:14.270459 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ee1127-ecda-4698-9324-31c1cd818725 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:22:14.270761 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ee1127-ecda-4698-9324-31c1cd818725 admin admin] PATH_INFO: `/v3/roles` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:22:14.271475 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ee1127-ecda-4698-9324-31c1cd818725 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:22:14.273677 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ee1127-ecda-4698-9324-31c1cd818725 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:22:14.284228 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1374/2727] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 10 12:22:14 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 12:22:14.288425 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a363cd-8d32-4e33-b465-5d658207dd3d admin admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:22:14.298421 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a363cd-8d32-4e33-b465-5d658207dd3d admin admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:22:14.303033 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144b557f-66e4-4b8a-a858-1135de8bc652 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:22:14.303693 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144b557f-66e4-4b8a-a858-1135de8bc652 admin admin] REQUEST_METHOD: `PUT` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:22:14.305016 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144b557f-66e4-4b8a-a858-1135de8bc652 admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:22:14.305295 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144b557f-66e4-4b8a-a858-1135de8bc652 admin admin] PATH_INFO: `/v3/projects/b5b324c81c7a482bbb61caae5eed59ce/users/cc77924c7f234cc2be9544a607058621/roles/0d822a0674b6495fac623877a3b38025` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:22:14.321391 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144b557f-66e4-4b8a-a858-1135de8bc652 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b5b324c81c7a482bbb61caae5eed59ce, user_id=cc77924c7f234cc2be9544a607058621, role_id=0d822a0674b6495fac623877a3b38025)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:22:14.324302 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144b557f-66e4-4b8a-a858-1135de8bc652 admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:22:14.404993 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1354/2728] 10.222.0.46 () {60 vars in 1483 bytes} [Mon Oct 10 12:22:14 2022] PUT /identity/v3/projects/b5b324c81c7a482bbb61caae5eed59ce/users/cc77924c7f234cc2be9544a607058621/roles/0d822a0674b6495fac623877a3b38025 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:22:14.409091 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ee1127-ecda-4698-9324-31c1cd818725 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:22:14.419262 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ee1127-ecda-4698-9324-31c1cd818725 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:22:14.477058 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49eb4466-c3a5-4b36-9c55-16f1578f347c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:22:14.477595 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49eb4466-c3a5-4b36-9c55-16f1578f347c admin admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:22:14.477879 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49eb4466-c3a5-4b36-9c55-16f1578f347c admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:22:14.478153 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49eb4466-c3a5-4b36-9c55-16f1578f347c admin admin] PATH_INFO: `/v3/roles` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:22:14.478888 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49eb4466-c3a5-4b36-9c55-16f1578f347c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:22:14.480485 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49eb4466-c3a5-4b36-9c55-16f1578f347c admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:22:14.494268 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1375/2729] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 10 12:22:14 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 12:22:14.495545 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144b557f-66e4-4b8a-a858-1135de8bc652 admin admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:22:14.506229 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144b557f-66e4-4b8a-a858-1135de8bc652 admin admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:22:14.512070 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc69f506-ae79-4ac1-bc99-78f10205df1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:22:14.512733 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc69f506-ae79-4ac1-bc99-78f10205df1f admin admin] REQUEST_METHOD: `PUT` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:22:14.513013 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc69f506-ae79-4ac1-bc99-78f10205df1f admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:22:14.513290 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc69f506-ae79-4ac1-bc99-78f10205df1f admin admin] PATH_INFO: `/v3/projects/b5b324c81c7a482bbb61caae5eed59ce/users/cc77924c7f234cc2be9544a607058621/roles/0d822a0674b6495fac623877a3b38025` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:22:14.515314 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc69f506-ae79-4ac1-bc99-78f10205df1f admin admin] RBAC: Authorizing `identity:create_grant(project_id=b5b324c81c7a482bbb61caae5eed59ce, user_id=cc77924c7f234cc2be9544a607058621, role_id=0d822a0674b6495fac623877a3b38025)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:22:14.518545 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc69f506-ae79-4ac1-bc99-78f10205df1f admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:22:14.530559 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1355/2730] 10.222.0.46 () {60 vars in 1483 bytes} [Mon Oct 10 12:22:14 2022] PUT /identity/v3/projects/b5b324c81c7a482bbb61caae5eed59ce/users/cc77924c7f234cc2be9544a607058621/roles/0d822a0674b6495fac623877a3b38025 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:22:14.542782 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7561d8be-3736-4ab6-8e6a-349961231348 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:22:14.543248 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7561d8be-3736-4ab6-8e6a-349961231348 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:22:14.543863 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7561d8be-3736-4ab6-8e6a-349961231348 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:22:14.557821 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:22:14.583406 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-7561d8be-3736-4ab6-8e6a-349961231348 None None] MFA Rules not processed for user `cc77924c7f234cc2be9544a607058621`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:22:14.584677 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-7561d8be-3736-4ab6-8e6a-349961231348 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:22:14.661948 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1376/2731] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:22:14 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:22:14.677942 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc69f506-ae79-4ac1-bc99-78f10205df1f admin admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:22:14.680413 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-cc69f506-ae79-4ac1-bc99-78f10205df1f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:22:14.746605 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc69f506-ae79-4ac1-bc99-78f10205df1f admin admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:22:14.750446 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914bde30-714b-4169-baa4-092ad1624908 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3ae093470484b93b23dcfae1657df9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad3772f7b6a47c28b7ba37a9316a3c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:22:14.751103 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914bde30-714b-4169-baa4-092ad1624908 service cinder] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:22:14.751437 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914bde30-714b-4169-baa4-092ad1624908 service cinder] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:22:14.751697 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914bde30-714b-4169-baa4-092ad1624908 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:22:14.762097 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914bde30-714b-4169-baa4-092ad1624908 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:22:14.763504 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914bde30-714b-4169-baa4-092ad1624908 service cinder] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:22:14.773793 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1356/2732] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 10 12:22:14 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:22:15.123920 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7561d8be-3736-4ab6-8e6a-349961231348 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:22:15.133490 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7561d8be-3736-4ab6-8e6a-349961231348 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:22:15.137246 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2223bf2-d3eb-43e2-9cd4-bb7ab818a81b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3ae093470484b93b23dcfae1657df9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad3772f7b6a47c28b7ba37a9316a3c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:22:15.137847 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2223bf2-d3eb-43e2-9cd4-bb7ab818a81b service cinder] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:22:15.138140 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2223bf2-d3eb-43e2-9cd4-bb7ab818a81b service cinder] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:22:15.138414 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2223bf2-d3eb-43e2-9cd4-bb7ab818a81b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:22:15.200811 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2223bf2-d3eb-43e2-9cd4-bb7ab818a81b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:22:15.202243 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2223bf2-d3eb-43e2-9cd4-bb7ab818a81b service cinder] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:22:15.212401 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1377/2733] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 10 12:22:15 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 10 12:22:18.080361 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914bde30-714b-4169-baa4-092ad1624908 service cinder] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:22:18.081813 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-914bde30-714b-4169-baa4-092ad1624908 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:22:18.153835 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914bde30-714b-4169-baa4-092ad1624908 service cinder] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:22:18.158896 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e44d12-2bca-4514-b529-93a2bec00dff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:22:18.159548 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e44d12-2bca-4514-b529-93a2bec00dff admin admin] REQUEST_METHOD: `DELETE` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:22:18.159828 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e44d12-2bca-4514-b529-93a2bec00dff admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:22:18.160104 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e44d12-2bca-4514-b529-93a2bec00dff admin admin] PATH_INFO: `/v3/users/09cf8764085d4a95bc4d515991ac2a2a` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:22:18.161265 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e44d12-2bca-4514-b529-93a2bec00dff admin admin] RBAC: Authorizing `identity:delete_user(user_id=09cf8764085d4a95bc4d515991ac2a2a)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:22:18.163355 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e44d12-2bca-4514-b529-93a2bec00dff admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:22:18.273417 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=90,keystone:INSERT=5,keystone:DELETE=5 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:22:18.371889 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-58e44d12-2bca-4514-b529-93a2bec00dff admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '09cf8764085d4a95bc4d515991ac2a2a'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:22:18.378613 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-58e44d12-2bca-4514-b529-93a2bec00dff admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '09cf8764085d4a95bc4d515991ac2a2a'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:22:18.384915 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-58e44d12-2bca-4514-b529-93a2bec00dff admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '09cf8764085d4a95bc4d515991ac2a2a'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:22:18.458155 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-58e44d12-2bca-4514-b529-93a2bec00dff admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '09cf8764085d4a95bc4d515991ac2a2a'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:22:18.474362 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-58e44d12-2bca-4514-b529-93a2bec00dff admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '09cf8764085d4a95bc4d515991ac2a2a'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:22:18.481116 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1357/2734] 10.222.0.46 () {58 vars in 1224 bytes} [Mon Oct 10 12:22:18 2022] DELETE /identity/v3/users/09cf8764085d4a95bc4d515991ac2a2a => generated 0 bytes in 401 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:22:18.488239 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8648d867-7cd3-42e2-a201-7593818f1428 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:22:18.488239 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8648d867-7cd3-42e2-a201-7593818f1428 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:22:18.488239 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8648d867-7cd3-42e2-a201-7593818f1428 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:22:18.505610 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-8648d867-7cd3-42e2-a201-7593818f1428 None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:22:18.506829 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-8648d867-7cd3-42e2-a201-7593818f1428 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:22:18.570284 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1378/2735] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:22:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 10 12:22:18.580757 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e44d12-2bca-4514-b529-93a2bec00dff admin admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:22:18.582318 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-58e44d12-2bca-4514-b529-93a2bec00dff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:22:18.645975 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e44d12-2bca-4514-b529-93a2bec00dff admin admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:22:18.650175 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2dcdc8-9a48-4738-a5b2-c4b15c95b21c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58b1c7c85302421cba823ace4201861f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad3772f7b6a47c28b7ba37a9316a3c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:22:18.654310 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2dcdc8-9a48-4738-a5b2-c4b15c95b21c service neutron] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:22:18.654310 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2dcdc8-9a48-4738-a5b2-c4b15c95b21c service neutron] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:22:18.654310 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2dcdc8-9a48-4738-a5b2-c4b15c95b21c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:22:18.661312 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2dcdc8-9a48-4738-a5b2-c4b15c95b21c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:22:18.662821 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2dcdc8-9a48-4738-a5b2-c4b15c95b21c service neutron] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:22:18.672868 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1358/2736] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 10 12:22:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:22:19.126048 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8648d867-7cd3-42e2-a201-7593818f1428 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:22:19.127650 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-8648d867-7cd3-42e2-a201-7593818f1428 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:22:19.141001 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8648d867-7cd3-42e2-a201-7593818f1428 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:22:19.146287 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e70b094-4581-4151-bf54-b4e2412ea065 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:22:19.146962 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e70b094-4581-4151-bf54-b4e2412ea065 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:22:19.147240 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e70b094-4581-4151-bf54-b4e2412ea065 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:22:19.147511 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e70b094-4581-4151-bf54-b4e2412ea065 admin admin] PATH_INFO: `/v3/projects/8ef95c3116be40ebadade4cb2175948e` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:22:19.148624 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e70b094-4581-4151-bf54-b4e2412ea065 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8ef95c3116be40ebadade4cb2175948e)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:22:19.151283 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e70b094-4581-4151-bf54-b4e2412ea065 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:22:19.182691 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=100 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:22:19.267925 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-8e70b094-4581-4151-bf54-b4e2412ea065 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8ef95c3116be40ebadade4cb2175948e', 'request_id': 'req-8e70b094-4581-4151-bf54-b4e2412ea065'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:22:19.273523 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-8e70b094-4581-4151-bf54-b4e2412ea065 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8ef95c3116be40ebadade4cb2175948e', 'request_id': 'req-8e70b094-4581-4151-bf54-b4e2412ea065'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:22:19.280289 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-8e70b094-4581-4151-bf54-b4e2412ea065 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8ef95c3116be40ebadade4cb2175948e', 'request_id': 'req-8e70b094-4581-4151-bf54-b4e2412ea065'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:22:19.360818 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-8e70b094-4581-4151-bf54-b4e2412ea065 admin admin] The token cache is being invalidate because project 8ef95c3116be40ebadade4cb2175948e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81677) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 10 12:22:19.361106 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-8e70b094-4581-4151-bf54-b4e2412ea065 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:22:19.361781 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-8e70b094-4581-4151-bf54-b4e2412ea065 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:22:19.363354 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1379/2737] 10.222.0.46 () {58 vars in 1233 bytes} [Mon Oct 10 12:22:19 2022] DELETE /identity/v3/projects/8ef95c3116be40ebadade4cb2175948e => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:22:19.367422 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2dcdc8-9a48-4738-a5b2-c4b15c95b21c service neutron] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:22:19.377829 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2dcdc8-9a48-4738-a5b2-c4b15c95b21c service neutron] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:22:19.433845 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90a2464-8adf-4801-845f-29ec6957edbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:22:19.434386 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90a2464-8adf-4801-845f-29ec6957edbf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:22:19.434662 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90a2464-8adf-4801-845f-29ec6957edbf admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:22:19.434959 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90a2464-8adf-4801-845f-29ec6957edbf admin admin] PATH_INFO: `/v3/users/41bd1e7b92dd468ab948e52f1ec3c406` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:22:19.436101 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90a2464-8adf-4801-845f-29ec6957edbf admin admin] RBAC: Authorizing `identity:delete_user(user_id=41bd1e7b92dd468ab948e52f1ec3c406)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:22:19.438000 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90a2464-8adf-4801-845f-29ec6957edbf admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:22:19.676128 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-f90a2464-8adf-4801-845f-29ec6957edbf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '41bd1e7b92dd468ab948e52f1ec3c406'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:22:19.678495 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:22:19.684725 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-f90a2464-8adf-4801-845f-29ec6957edbf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '41bd1e7b92dd468ab948e52f1ec3c406'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:22:19.685501 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-f90a2464-8adf-4801-845f-29ec6957edbf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '41bd1e7b92dd468ab948e52f1ec3c406'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:22:19.836895 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-f90a2464-8adf-4801-845f-29ec6957edbf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '41bd1e7b92dd468ab948e52f1ec3c406'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:22:19.851535 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-f90a2464-8adf-4801-845f-29ec6957edbf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '41bd1e7b92dd468ab948e52f1ec3c406'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:22:19.861729 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1359/2738] 10.222.0.46 () {58 vars in 1224 bytes} [Mon Oct 10 12:22:19 2022] DELETE /identity/v3/users/41bd1e7b92dd468ab948e52f1ec3c406 => generated 0 bytes in 491 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:22:20.277650 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e70b094-4581-4151-bf54-b4e2412ea065 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:22:20.282687 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-8e70b094-4581-4151-bf54-b4e2412ea065 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:22:20.296206 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e70b094-4581-4151-bf54-b4e2412ea065 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:22:20.380028 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e1671d-2999-44b8-97b9-62fc3d4ae715 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:22:20.380681 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e1671d-2999-44b8-97b9-62fc3d4ae715 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:22:20.380980 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e1671d-2999-44b8-97b9-62fc3d4ae715 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:22:20.381242 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e1671d-2999-44b8-97b9-62fc3d4ae715 admin admin] PATH_INFO: `/v3/projects/9d26079a048f44ffab880a063bcec589` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:22:20.382383 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e1671d-2999-44b8-97b9-62fc3d4ae715 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9d26079a048f44ffab880a063bcec589)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:22:20.384484 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e1671d-2999-44b8-97b9-62fc3d4ae715 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:22:20.492908 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-11e1671d-2999-44b8-97b9-62fc3d4ae715 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9d26079a048f44ffab880a063bcec589', 'request_id': 'req-11e1671d-2999-44b8-97b9-62fc3d4ae715'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:22:20.498957 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-11e1671d-2999-44b8-97b9-62fc3d4ae715 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9d26079a048f44ffab880a063bcec589', 'request_id': 'req-11e1671d-2999-44b8-97b9-62fc3d4ae715'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:22:20.505789 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-11e1671d-2999-44b8-97b9-62fc3d4ae715 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9d26079a048f44ffab880a063bcec589', 'request_id': 'req-11e1671d-2999-44b8-97b9-62fc3d4ae715'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:22:20.612257 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-11e1671d-2999-44b8-97b9-62fc3d4ae715 admin admin] The token cache is being invalidate because project 9d26079a048f44ffab880a063bcec589 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81677) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 10 12:22:20.612627 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-11e1671d-2999-44b8-97b9-62fc3d4ae715 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:22:20.613295 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-11e1671d-2999-44b8-97b9-62fc3d4ae715 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:22:20.614895 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1380/2739] 10.222.0.46 () {58 vars in 1233 bytes} [Mon Oct 10 12:22:20 2022] DELETE /identity/v3/projects/9d26079a048f44ffab880a063bcec589 => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:22:20.620355 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c2cacc-08c9-416d-81ad-140c2170cba2 None None] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:22:20.620865 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c2cacc-08c9-416d-81ad-140c2170cba2 None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:22:20.620983 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c2cacc-08c9-416d-81ad-140c2170cba2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:22:20.640884 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.auth.core [None req-f5c2cacc-08c9-416d-81ad-140c2170cba2 None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:22:20.642099 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-f5c2cacc-08c9-416d-81ad-140c2170cba2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:22:20.704178 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1360/2740] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:22:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:22:20.712893 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb523e94-c584-449d-a59d-2286ed93b4e5 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:22:20.713179 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb523e94-c584-449d-a59d-2286ed93b4e5 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:22:20.713460 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb523e94-c584-449d-a59d-2286ed93b4e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:22:20.735250 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-bb523e94-c584-449d-a59d-2286ed93b4e5 None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:22:20.736535 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-bb523e94-c584-449d-a59d-2286ed93b4e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:22:20.745332 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1381/2741] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:22:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:22:20.749908 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c2cacc-08c9-416d-81ad-140c2170cba2 None None] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:22:20.760578 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c2cacc-08c9-416d-81ad-140c2170cba2 None None] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:22:20.765641 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f1e7ef-7dcd-41b4-8834-0dd85a359d8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:22:20.770303 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f1e7ef-7dcd-41b4-8834-0dd85a359d8a admin admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:22:20.770303 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f1e7ef-7dcd-41b4-8834-0dd85a359d8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:22:20.770303 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f1e7ef-7dcd-41b4-8834-0dd85a359d8a admin admin] PATH_INFO: `/v3/domains` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:22:20.770303 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f1e7ef-7dcd-41b4-8834-0dd85a359d8a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 12:22:20.770303 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f1e7ef-7dcd-41b4-8834-0dd85a359d8a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:22:20.770303 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f1e7ef-7dcd-41b4-8834-0dd85a359d8a admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:22:20.782076 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1361/2742] 10.222.0.46 () {58 vars in 1153 bytes} [Mon Oct 10 12:22:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 12:22:20.786465 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb523e94-c584-449d-a59d-2286ed93b4e5 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:22:20.797407 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb523e94-c584-449d-a59d-2286ed93b4e5 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:22:20.802318 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c34e8c2-8179-454e-a213-9924fcd91fda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:22:20.806330 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c34e8c2-8179-454e-a213-9924fcd91fda admin admin] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:22:20.806330 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c34e8c2-8179-454e-a213-9924fcd91fda admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:22:20.806330 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c34e8c2-8179-454e-a213-9924fcd91fda admin admin] PATH_INFO: `/v3/projects` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:22:20.806330 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c34e8c2-8179-454e-a213-9924fcd91fda admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:22:20.806769 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c34e8c2-8179-454e-a213-9924fcd91fda admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:22:20.917930 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1382/2743] 10.222.0.46 () {60 vars in 1153 bytes} [Mon Oct 10 12:22:20 2022] POST /identity/v3/projects => generated 389 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 12:22:20.922283 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f1e7ef-7dcd-41b4-8834-0dd85a359d8a admin admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:22:20.933658 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f1e7ef-7dcd-41b4-8834-0dd85a359d8a admin admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:22:20.996696 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f133e8b-f404-47f1-ac0f-b07852f19641 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:22:20.997308 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f133e8b-f404-47f1-ac0f-b07852f19641 admin admin] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:22:20.997646 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f133e8b-f404-47f1-ac0f-b07852f19641 admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:22:20.997848 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f133e8b-f404-47f1-ac0f-b07852f19641 admin admin] PATH_INFO: `/v3/users` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:22:20.998478 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f133e8b-f404-47f1-ac0f-b07852f19641 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:22:21.000416 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f133e8b-f404-47f1-ac0f-b07852f19641 admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:22:21.110855 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1362/2744] 10.222.0.46 () {60 vars in 1144 bytes} [Mon Oct 10 12:22:20 2022] POST /identity/v3/users => generated 295 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 12:22:21.111930 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c34e8c2-8179-454e-a213-9924fcd91fda admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:22:21.122563 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c34e8c2-8179-454e-a213-9924fcd91fda admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:22:21.127548 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafae744-cfbe-4bee-9c0b-fb59e73ca050 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:22:21.128219 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafae744-cfbe-4bee-9c0b-fb59e73ca050 admin admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:22:21.128433 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafae744-cfbe-4bee-9c0b-fb59e73ca050 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:22:21.128709 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafae744-cfbe-4bee-9c0b-fb59e73ca050 admin admin] PATH_INFO: `/v3/roles` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:22:21.129364 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cafae744-cfbe-4bee-9c0b-fb59e73ca050 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:22:21.130977 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cafae744-cfbe-4bee-9c0b-fb59e73ca050 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:22:21.137227 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:DELETE=14,keystone:SELECT=83,keystone:INSERT=3 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:22:21.145569 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1383/2745] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 10 12:22:21 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 12:22:21.146698 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f133e8b-f404-47f1-ac0f-b07852f19641 admin admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:22:21.157595 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f133e8b-f404-47f1-ac0f-b07852f19641 admin admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:22:21.162492 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fa0409-cd6f-478f-bb93-198bbe1b2fbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:22:21.163129 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fa0409-cd6f-478f-bb93-198bbe1b2fbd admin admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:22:21.163413 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fa0409-cd6f-478f-bb93-198bbe1b2fbd admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:22:21.163718 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fa0409-cd6f-478f-bb93-198bbe1b2fbd admin admin] PATH_INFO: `/v3/roles` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:22:21.164424 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19fa0409-cd6f-478f-bb93-198bbe1b2fbd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:22:21.166250 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19fa0409-cd6f-478f-bb93-198bbe1b2fbd admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:22:21.178020 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1363/2746] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 10 12:22:21 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 12:22:21.182941 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafae744-cfbe-4bee-9c0b-fb59e73ca050 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:22:21.193240 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafae744-cfbe-4bee-9c0b-fb59e73ca050 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:22:21.199076 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146d163f-43f0-46e6-989f-5a0a3638b97f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:22:21.201563 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146d163f-43f0-46e6-989f-5a0a3638b97f admin admin] REQUEST_METHOD: `PUT` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:22:21.201563 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146d163f-43f0-46e6-989f-5a0a3638b97f admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:22:21.201563 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146d163f-43f0-46e6-989f-5a0a3638b97f admin admin] PATH_INFO: `/v3/projects/d4dee8fb62c04681a4355cac6272626d/users/cc7dbc8a2af845d0a352d4de83766a64/roles/b2cb2a6e818e47338ce071e6fedfc673` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:22:21.215615 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146d163f-43f0-46e6-989f-5a0a3638b97f admin admin] RBAC: Authorizing `identity:create_grant(project_id=d4dee8fb62c04681a4355cac6272626d, user_id=cc7dbc8a2af845d0a352d4de83766a64, role_id=b2cb2a6e818e47338ce071e6fedfc673)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:22:21.218244 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146d163f-43f0-46e6-989f-5a0a3638b97f admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:22:21.307264 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1384/2747] 10.222.0.46 () {60 vars in 1483 bytes} [Mon Oct 10 12:22:21 2022] PUT /identity/v3/projects/d4dee8fb62c04681a4355cac6272626d/users/cc7dbc8a2af845d0a352d4de83766a64/roles/b2cb2a6e818e47338ce071e6fedfc673 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:22:21.317941 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d46bce-e18a-4231-a6cf-1e43d407e32f None None] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:22:21.318239 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d46bce-e18a-4231-a6cf-1e43d407e32f None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:22:21.318519 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d46bce-e18a-4231-a6cf-1e43d407e32f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:22:21.346902 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.auth.core [None req-02d46bce-e18a-4231-a6cf-1e43d407e32f None None] MFA Rules not processed for user `cc7dbc8a2af845d0a352d4de83766a64`. Rule list: `[]` (Enabled: `True`). {{(pid=81676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:22:21.348102 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-02d46bce-e18a-4231-a6cf-1e43d407e32f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:22:21.369368 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:22:21.417624 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1364/2748] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:22:21 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:22:21.423021 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146d163f-43f0-46e6-989f-5a0a3638b97f admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:22:21.493203 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146d163f-43f0-46e6-989f-5a0a3638b97f admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:22:21.498078 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc15147-b67d-46f5-ad09-e2cc90a7a47f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:22:21.498723 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc15147-b67d-46f5-ad09-e2cc90a7a47f admin admin] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:22:21.499008 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc15147-b67d-46f5-ad09-e2cc90a7a47f admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:22:21.499288 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc15147-b67d-46f5-ad09-e2cc90a7a47f admin admin] PATH_INFO: `/v3/projects` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:22:21.499957 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc15147-b67d-46f5-ad09-e2cc90a7a47f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:22:21.501886 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc15147-b67d-46f5-ad09-e2cc90a7a47f admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:22:21.609132 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1385/2749] 10.222.0.46 () {60 vars in 1153 bytes} [Mon Oct 10 12:22:21 2022] POST /identity/v3/projects => generated 389 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 12:22:21.613254 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d46bce-e18a-4231-a6cf-1e43d407e32f None None] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:22:21.627766 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d46bce-e18a-4231-a6cf-1e43d407e32f None None] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:22:21.682222 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3387ffc0-4d07-4221-ba04-5d7b3eb7cc9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:22:21.682867 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3387ffc0-4d07-4221-ba04-5d7b3eb7cc9c admin admin] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:22:21.683152 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3387ffc0-4d07-4221-ba04-5d7b3eb7cc9c admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:22:21.683429 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3387ffc0-4d07-4221-ba04-5d7b3eb7cc9c admin admin] PATH_INFO: `/v3/users` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:22:21.684064 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3387ffc0-4d07-4221-ba04-5d7b3eb7cc9c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:22:21.685992 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3387ffc0-4d07-4221-ba04-5d7b3eb7cc9c admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:22:21.783317 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1365/2750] 10.222.0.46 () {60 vars in 1144 bytes} [Mon Oct 10 12:22:21 2022] POST /identity/v3/users => generated 301 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 12:22:21.787327 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc15147-b67d-46f5-ad09-e2cc90a7a47f admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:22:21.798018 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc15147-b67d-46f5-ad09-e2cc90a7a47f admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:22:21.803416 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150072ee-a8a2-4e96-bf02-73158f3ca993 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:22:21.804032 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150072ee-a8a2-4e96-bf02-73158f3ca993 admin admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:22:21.804319 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150072ee-a8a2-4e96-bf02-73158f3ca993 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:22:21.804599 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150072ee-a8a2-4e96-bf02-73158f3ca993 admin admin] PATH_INFO: `/v3/roles` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:22:21.805244 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-150072ee-a8a2-4e96-bf02-73158f3ca993 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:22:21.806943 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-150072ee-a8a2-4e96-bf02-73158f3ca993 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:22:21.818272 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1386/2751] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 10 12:22:21 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 12:22:21.823180 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3387ffc0-4d07-4221-ba04-5d7b3eb7cc9c admin admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:22:21.833788 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3387ffc0-4d07-4221-ba04-5d7b3eb7cc9c admin admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:22:21.839668 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05ca03f-121f-4d44-97d9-781f56f6e5ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:22:21.840331 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05ca03f-121f-4d44-97d9-781f56f6e5ee admin admin] REQUEST_METHOD: `PUT` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:22:21.840614 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05ca03f-121f-4d44-97d9-781f56f6e5ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:22:21.840893 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05ca03f-121f-4d44-97d9-781f56f6e5ee admin admin] PATH_INFO: `/v3/projects/f9a0cee1207245788a7b144371a869a6/users/3e18d15d3cab4fc2896d049444227986/roles/0d822a0674b6495fac623877a3b38025` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:22:21.856086 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05ca03f-121f-4d44-97d9-781f56f6e5ee admin admin] RBAC: Authorizing `identity:create_grant(project_id=f9a0cee1207245788a7b144371a869a6, user_id=3e18d15d3cab4fc2896d049444227986, role_id=0d822a0674b6495fac623877a3b38025)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:22:21.858791 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05ca03f-121f-4d44-97d9-781f56f6e5ee admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:22:21.950705 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1366/2752] 10.222.0.46 () {60 vars in 1483 bytes} [Mon Oct 10 12:22:21 2022] PUT /identity/v3/projects/f9a0cee1207245788a7b144371a869a6/users/3e18d15d3cab4fc2896d049444227986/roles/0d822a0674b6495fac623877a3b38025 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:22:21.954604 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150072ee-a8a2-4e96-bf02-73158f3ca993 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:22:21.965020 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150072ee-a8a2-4e96-bf02-73158f3ca993 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:22:22.021969 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5159e8-9bb0-4ae7-85d3-15e945b618d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:22:22.022571 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5159e8-9bb0-4ae7-85d3-15e945b618d7 admin admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:22:22.022879 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5159e8-9bb0-4ae7-85d3-15e945b618d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:22:22.023150 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5159e8-9bb0-4ae7-85d3-15e945b618d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:22:22.023774 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5159e8-9bb0-4ae7-85d3-15e945b618d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:22:22.025390 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5159e8-9bb0-4ae7-85d3-15e945b618d7 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:22:22.035051 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1387/2753] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 10 12:22:21 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 12:22:22.039264 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05ca03f-121f-4d44-97d9-781f56f6e5ee admin admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:22:22.052852 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05ca03f-121f-4d44-97d9-781f56f6e5ee admin admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:22:22.057922 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48999ed-4df5-4320-a511-b11e09624da7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:22:22.058580 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48999ed-4df5-4320-a511-b11e09624da7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:22:22.058886 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48999ed-4df5-4320-a511-b11e09624da7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:22:22.059161 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48999ed-4df5-4320-a511-b11e09624da7 admin admin] PATH_INFO: `/v3/projects/f9a0cee1207245788a7b144371a869a6/users/3e18d15d3cab4fc2896d049444227986/roles/0d822a0674b6495fac623877a3b38025` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:22:22.061381 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48999ed-4df5-4320-a511-b11e09624da7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f9a0cee1207245788a7b144371a869a6, user_id=3e18d15d3cab4fc2896d049444227986, role_id=0d822a0674b6495fac623877a3b38025)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:22:22.064059 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48999ed-4df5-4320-a511-b11e09624da7 admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:22:22.076607 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1367/2754] 10.222.0.46 () {60 vars in 1483 bytes} [Mon Oct 10 12:22:22 2022] PUT /identity/v3/projects/f9a0cee1207245788a7b144371a869a6/users/3e18d15d3cab4fc2896d049444227986/roles/0d822a0674b6495fac623877a3b38025 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:22:22.142982 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaeb881e-97e3-4728-b4ef-c97d430d0aa9 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:22:22.143280 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaeb881e-97e3-4728-b4ef-c97d430d0aa9 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:22:22.143558 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaeb881e-97e3-4728-b4ef-c97d430d0aa9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:22:22.173089 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-eaeb881e-97e3-4728-b4ef-c97d430d0aa9 None None] MFA Rules not processed for user `3e18d15d3cab4fc2896d049444227986`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:22:22.174233 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-eaeb881e-97e3-4728-b4ef-c97d430d0aa9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:22:22.209693 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:22:22.246357 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1388/2755] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:22:22 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:22:22.257378 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48999ed-4df5-4320-a511-b11e09624da7 admin admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:22:22.259145 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-d48999ed-4df5-4320-a511-b11e09624da7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:22:22.317057 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48999ed-4df5-4320-a511-b11e09624da7 admin admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:22:22.320775 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f667f9-4f74-4db4-8067-901664422b5a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3ae093470484b93b23dcfae1657df9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad3772f7b6a47c28b7ba37a9316a3c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:22:22.321401 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f667f9-4f74-4db4-8067-901664422b5a service cinder] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:22:22.321676 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f667f9-4f74-4db4-8067-901664422b5a service cinder] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:22:22.321948 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f667f9-4f74-4db4-8067-901664422b5a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:22:22.330913 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f667f9-4f74-4db4-8067-901664422b5a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:22:22.332339 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f667f9-4f74-4db4-8067-901664422b5a service cinder] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:22:22.342019 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1368/2756] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 10 12:22:22 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:22:22.480290 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaeb881e-97e3-4728-b4ef-c97d430d0aa9 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:22:22.542563 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaeb881e-97e3-4728-b4ef-c97d430d0aa9 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:22:22.547113 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7eceef-41f9-4191-950a-60acee3934c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:22:22.547796 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7eceef-41f9-4191-950a-60acee3934c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:22:22.548141 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7eceef-41f9-4191-950a-60acee3934c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:22:22.548426 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7eceef-41f9-4191-950a-60acee3934c5 admin admin] PATH_INFO: `/v3/users/cc7dbc8a2af845d0a352d4de83766a64` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:22:22.549554 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7eceef-41f9-4191-950a-60acee3934c5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cc7dbc8a2af845d0a352d4de83766a64)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:22:22.551767 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7eceef-41f9-4191-950a-60acee3934c5 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:22:22.763882 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-6e7eceef-41f9-4191-950a-60acee3934c5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cc7dbc8a2af845d0a352d4de83766a64'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:22:22.770737 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-6e7eceef-41f9-4191-950a-60acee3934c5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cc7dbc8a2af845d0a352d4de83766a64'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:22:22.771404 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-6e7eceef-41f9-4191-950a-60acee3934c5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cc7dbc8a2af845d0a352d4de83766a64'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:22:22.838640 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-6e7eceef-41f9-4191-950a-60acee3934c5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cc7dbc8a2af845d0a352d4de83766a64'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:22:22.851043 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-6e7eceef-41f9-4191-950a-60acee3934c5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cc7dbc8a2af845d0a352d4de83766a64'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:22:22.857550 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1389/2757] 10.222.0.46 () {58 vars in 1224 bytes} [Mon Oct 10 12:22:22 2022] DELETE /identity/v3/users/cc7dbc8a2af845d0a352d4de83766a64 => generated 0 bytes in 378 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:22:22.862189 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409497a0-19de-4b44-8a2a-9584e1196d80 None None] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:22:22.862698 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409497a0-19de-4b44-8a2a-9584e1196d80 None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:22:22.862805 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409497a0-19de-4b44-8a2a-9584e1196d80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:22:22.881542 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.auth.core [None req-409497a0-19de-4b44-8a2a-9584e1196d80 None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:22:22.882771 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-409497a0-19de-4b44-8a2a-9584e1196d80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:22:22.894378 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:22:22.942862 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1369/2758] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:22:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:22:22.952461 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7eceef-41f9-4191-950a-60acee3934c5 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:22:22.954028 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-6e7eceef-41f9-4191-950a-60acee3934c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:22:23.011680 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7eceef-41f9-4191-950a-60acee3934c5 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:22:23.015508 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29219613-8235-4c6b-95bd-6554b03bf406 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58b1c7c85302421cba823ace4201861f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad3772f7b6a47c28b7ba37a9316a3c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:22:23.016056 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29219613-8235-4c6b-95bd-6554b03bf406 service neutron] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:22:23.016388 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29219613-8235-4c6b-95bd-6554b03bf406 service neutron] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:22:23.016607 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29219613-8235-4c6b-95bd-6554b03bf406 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:22:23.022679 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:22:23.027326 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29219613-8235-4c6b-95bd-6554b03bf406 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:22:23.028702 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29219613-8235-4c6b-95bd-6554b03bf406 service neutron] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:22:23.038512 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1390/2759] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 10 12:22:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:22:23.788329 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409497a0-19de-4b44-8a2a-9584e1196d80 None None] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:22:23.790380 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-409497a0-19de-4b44-8a2a-9584e1196d80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:22:23.803497 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409497a0-19de-4b44-8a2a-9584e1196d80 None None] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:22:23.811857 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c15961-5c81-484e-a8e9-c553e00fbfaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:22:23.812551 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c15961-5c81-484e-a8e9-c553e00fbfaf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:22:23.812837 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c15961-5c81-484e-a8e9-c553e00fbfaf admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:22:23.813117 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c15961-5c81-484e-a8e9-c553e00fbfaf admin admin] PATH_INFO: `/v3/projects/d4dee8fb62c04681a4355cac6272626d` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:22:23.814335 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c15961-5c81-484e-a8e9-c553e00fbfaf admin admin] RBAC: Authorizing `identity:delete_project(project_id=d4dee8fb62c04681a4355cac6272626d)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:22:23.816746 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c15961-5c81-484e-a8e9-c553e00fbfaf admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:22:23.925716 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-d3c15961-5c81-484e-a8e9-c553e00fbfaf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd4dee8fb62c04681a4355cac6272626d', 'request_id': 'req-d3c15961-5c81-484e-a8e9-c553e00fbfaf'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:22:23.932371 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-d3c15961-5c81-484e-a8e9-c553e00fbfaf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd4dee8fb62c04681a4355cac6272626d', 'request_id': 'req-d3c15961-5c81-484e-a8e9-c553e00fbfaf'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:22:23.938887 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-d3c15961-5c81-484e-a8e9-c553e00fbfaf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd4dee8fb62c04681a4355cac6272626d', 'request_id': 'req-d3c15961-5c81-484e-a8e9-c553e00fbfaf'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:22:23.998229 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-d3c15961-5c81-484e-a8e9-c553e00fbfaf admin admin] The token cache is being invalidate because project d4dee8fb62c04681a4355cac6272626d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81676) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 10 12:22:23.998515 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-d3c15961-5c81-484e-a8e9-c553e00fbfaf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:22:23.999223 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-d3c15961-5c81-484e-a8e9-c553e00fbfaf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:22:24.002881 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1370/2760] 10.222.0.46 () {58 vars in 1233 bytes} [Mon Oct 10 12:22:23 2022] DELETE /identity/v3/projects/d4dee8fb62c04681a4355cac6272626d => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:22:24.005001 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29219613-8235-4c6b-95bd-6554b03bf406 service neutron] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:22:24.015294 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29219613-8235-4c6b-95bd-6554b03bf406 service neutron] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:22:24.073431 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31828212-1630-4c45-a19b-fea401ef14ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:22:24.073976 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31828212-1630-4c45-a19b-fea401ef14ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:22:24.074253 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31828212-1630-4c45-a19b-fea401ef14ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:22:24.074526 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31828212-1630-4c45-a19b-fea401ef14ea admin admin] PATH_INFO: `/v3/users/3e18d15d3cab4fc2896d049444227986` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:22:24.075661 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31828212-1630-4c45-a19b-fea401ef14ea admin admin] RBAC: Authorizing `identity:delete_user(user_id=3e18d15d3cab4fc2896d049444227986)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:22:24.077601 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31828212-1630-4c45-a19b-fea401ef14ea admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:22:24.256523 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-31828212-1630-4c45-a19b-fea401ef14ea admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3e18d15d3cab4fc2896d049444227986'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:22:24.264303 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-31828212-1630-4c45-a19b-fea401ef14ea admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3e18d15d3cab4fc2896d049444227986'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:22:24.264987 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-31828212-1630-4c45-a19b-fea401ef14ea admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3e18d15d3cab4fc2896d049444227986'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:22:24.338277 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-31828212-1630-4c45-a19b-fea401ef14ea admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3e18d15d3cab4fc2896d049444227986'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:22:24.351701 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-31828212-1630-4c45-a19b-fea401ef14ea admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3e18d15d3cab4fc2896d049444227986'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:22:24.357711 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1391/2761] 10.222.0.46 () {58 vars in 1224 bytes} [Mon Oct 10 12:22:24 2022] DELETE /identity/v3/users/3e18d15d3cab4fc2896d049444227986 => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:22:24.818163 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c15961-5c81-484e-a8e9-c553e00fbfaf admin admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:22:24.819810 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-d3c15961-5c81-484e-a8e9-c553e00fbfaf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:22:24.834163 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c15961-5c81-484e-a8e9-c553e00fbfaf admin admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:22:24.897678 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acce1760-8494-4752-b428-7e18a70ed2f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:22:24.898356 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acce1760-8494-4752-b428-7e18a70ed2f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:22:24.898638 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acce1760-8494-4752-b428-7e18a70ed2f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:22:24.899620 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acce1760-8494-4752-b428-7e18a70ed2f6 admin admin] PATH_INFO: `/v3/projects/f9a0cee1207245788a7b144371a869a6` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:22:24.900833 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acce1760-8494-4752-b428-7e18a70ed2f6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f9a0cee1207245788a7b144371a869a6)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:22:24.903049 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acce1760-8494-4752-b428-7e18a70ed2f6 admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:22:25.016937 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:22:25.021281 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-acce1760-8494-4752-b428-7e18a70ed2f6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f9a0cee1207245788a7b144371a869a6', 'request_id': 'req-acce1760-8494-4752-b428-7e18a70ed2f6'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:22:25.027799 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-acce1760-8494-4752-b428-7e18a70ed2f6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f9a0cee1207245788a7b144371a869a6', 'request_id': 'req-acce1760-8494-4752-b428-7e18a70ed2f6'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:22:25.034631 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-acce1760-8494-4752-b428-7e18a70ed2f6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f9a0cee1207245788a7b144371a869a6', 'request_id': 'req-acce1760-8494-4752-b428-7e18a70ed2f6'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:22:25.124560 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-acce1760-8494-4752-b428-7e18a70ed2f6 admin admin] The token cache is being invalidate because project f9a0cee1207245788a7b144371a869a6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81676) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 10 12:22:25.124894 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-acce1760-8494-4752-b428-7e18a70ed2f6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:22:25.125603 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-acce1760-8494-4752-b428-7e18a70ed2f6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:22:25.127438 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1371/2762] 10.222.0.46 () {58 vars in 1233 bytes} [Mon Oct 10 12:22:24 2022] DELETE /identity/v3/projects/f9a0cee1207245788a7b144371a869a6 => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:22:25.133076 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e4ae25-7939-407c-9908-41f0d6654b6c None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:22:25.134155 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e4ae25-7939-407c-9908-41f0d6654b6c None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:22:25.134279 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e4ae25-7939-407c-9908-41f0d6654b6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:22:25.156429 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-73e4ae25-7939-407c-9908-41f0d6654b6c None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:22:25.157639 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-73e4ae25-7939-407c-9908-41f0d6654b6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:22:25.218651 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1392/2763] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:22:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 10 12:22:25.226549 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50035ac3-5251-4670-92fe-a3278214077e None None] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:22:25.226854 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50035ac3-5251-4670-92fe-a3278214077e None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:22:25.227657 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50035ac3-5251-4670-92fe-a3278214077e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:22:25.248919 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.auth.core [None req-50035ac3-5251-4670-92fe-a3278214077e None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:22:25.250211 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-50035ac3-5251-4670-92fe-a3278214077e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:22:25.258310 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1372/2764] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:22:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:22:25.263084 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e4ae25-7939-407c-9908-41f0d6654b6c None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:22:25.273102 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e4ae25-7939-407c-9908-41f0d6654b6c None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:22:25.278507 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515e09f7-9f67-4c80-ad58-24875c10c191 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:22:25.279174 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515e09f7-9f67-4c80-ad58-24875c10c191 admin admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:22:25.279469 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515e09f7-9f67-4c80-ad58-24875c10c191 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:22:25.279749 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515e09f7-9f67-4c80-ad58-24875c10c191 admin admin] PATH_INFO: `/v3/domains` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:22:25.280401 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515e09f7-9f67-4c80-ad58-24875c10c191 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 12:22:25.280707 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515e09f7-9f67-4c80-ad58-24875c10c191 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:22:25.282301 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515e09f7-9f67-4c80-ad58-24875c10c191 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:22:25.291874 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:22:25.295230 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1393/2765] 10.222.0.46 () {58 vars in 1153 bytes} [Mon Oct 10 12:22:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 12:22:25.299617 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50035ac3-5251-4670-92fe-a3278214077e None None] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:22:25.309419 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50035ac3-5251-4670-92fe-a3278214077e None None] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:22:25.314077 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e7f9ec-d057-4f8c-a110-57eea45708ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:22:25.314702 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e7f9ec-d057-4f8c-a110-57eea45708ed admin admin] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:22:25.314993 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e7f9ec-d057-4f8c-a110-57eea45708ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:22:25.315267 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e7f9ec-d057-4f8c-a110-57eea45708ed admin admin] PATH_INFO: `/v3/projects` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:22:25.315907 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e7f9ec-d057-4f8c-a110-57eea45708ed admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:22:25.317836 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e7f9ec-d057-4f8c-a110-57eea45708ed admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:22:25.434501 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1373/2766] 10.222.0.46 () {60 vars in 1153 bytes} [Mon Oct 10 12:22:25 2022] POST /identity/v3/projects => generated 369 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 12:22:25.438696 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515e09f7-9f67-4c80-ad58-24875c10c191 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:22:25.449284 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515e09f7-9f67-4c80-ad58-24875c10c191 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:22:25.514263 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df70034-88ac-47b0-8084-5b5573e27200 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:22:25.514509 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df70034-88ac-47b0-8084-5b5573e27200 admin admin] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:22:25.514831 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df70034-88ac-47b0-8084-5b5573e27200 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:22:25.515109 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df70034-88ac-47b0-8084-5b5573e27200 admin admin] PATH_INFO: `/v3/users` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:22:25.515759 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df70034-88ac-47b0-8084-5b5573e27200 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:22:25.517863 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df70034-88ac-47b0-8084-5b5573e27200 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:22:25.629813 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1394/2767] 10.222.0.46 () {60 vars in 1144 bytes} [Mon Oct 10 12:22:25 2022] POST /identity/v3/users => generated 285 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 12:22:25.634827 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e7f9ec-d057-4f8c-a110-57eea45708ed admin admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:22:25.645310 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e7f9ec-d057-4f8c-a110-57eea45708ed admin admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:22:25.649976 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5133b299-7788-4a04-a3ac-af72dc878b54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:22:25.650622 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5133b299-7788-4a04-a3ac-af72dc878b54 admin admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:22:25.650938 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5133b299-7788-4a04-a3ac-af72dc878b54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:22:25.651216 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5133b299-7788-4a04-a3ac-af72dc878b54 admin admin] PATH_INFO: `/v3/roles` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:22:25.651864 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5133b299-7788-4a04-a3ac-af72dc878b54 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:22:25.653477 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5133b299-7788-4a04-a3ac-af72dc878b54 admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:22:25.663553 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1374/2768] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 10 12:22:25 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 12:22:25.667774 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df70034-88ac-47b0-8084-5b5573e27200 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:22:25.678305 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df70034-88ac-47b0-8084-5b5573e27200 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:22:25.683682 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf449c0-9906-4a1d-bcc2-3108d53dfc99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:22:25.684265 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf449c0-9906-4a1d-bcc2-3108d53dfc99 admin admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:22:25.684425 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf449c0-9906-4a1d-bcc2-3108d53dfc99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:22:25.684858 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf449c0-9906-4a1d-bcc2-3108d53dfc99 admin admin] PATH_INFO: `/v3/roles` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:22:25.685328 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf449c0-9906-4a1d-bcc2-3108d53dfc99 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:22:25.687090 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf449c0-9906-4a1d-bcc2-3108d53dfc99 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:22:25.699016 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1395/2769] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 10 12:22:25 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 12:22:25.703460 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5133b299-7788-4a04-a3ac-af72dc878b54 admin admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:22:25.716508 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5133b299-7788-4a04-a3ac-af72dc878b54 admin admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:22:25.724165 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fac800-34ea-4c75-a376-3d0e50c0290d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:22:25.724165 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fac800-34ea-4c75-a376-3d0e50c0290d admin admin] REQUEST_METHOD: `PUT` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:22:25.724165 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fac800-34ea-4c75-a376-3d0e50c0290d admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:22:25.724165 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fac800-34ea-4c75-a376-3d0e50c0290d admin admin] PATH_INFO: `/v3/projects/759a4fa2a6994b6892030c4d9c46d7c8/users/17c0750e56bc424fa46b2d2dd3bac023/roles/b2cb2a6e818e47338ce071e6fedfc673` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:22:25.741082 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1fac800-34ea-4c75-a376-3d0e50c0290d admin admin] RBAC: Authorizing `identity:create_grant(project_id=759a4fa2a6994b6892030c4d9c46d7c8, user_id=17c0750e56bc424fa46b2d2dd3bac023, role_id=b2cb2a6e818e47338ce071e6fedfc673)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:22:25.744308 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1fac800-34ea-4c75-a376-3d0e50c0290d admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:22:25.831298 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1375/2770] 10.222.0.46 () {60 vars in 1483 bytes} [Mon Oct 10 12:22:25 2022] PUT /identity/v3/projects/759a4fa2a6994b6892030c4d9c46d7c8/users/17c0750e56bc424fa46b2d2dd3bac023/roles/b2cb2a6e818e47338ce071e6fedfc673 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 10 12:22:25.842761 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805909b2-15a9-4370-be93-4509711bf355 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:22:25.843052 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805909b2-15a9-4370-be93-4509711bf355 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:22:25.843375 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805909b2-15a9-4370-be93-4509711bf355 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:22:25.856110 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057d78e4-2050-4deb-9d03-c4c071c1bdca None None] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:22:25.856482 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057d78e4-2050-4deb-9d03-c4c071c1bdca None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:22:25.856835 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057d78e4-2050-4deb-9d03-c4c071c1bdca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:22:25.875651 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-805909b2-15a9-4370-be93-4509711bf355 None None] MFA Rules not processed for user `17c0750e56bc424fa46b2d2dd3bac023`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:22:25.876993 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-805909b2-15a9-4370-be93-4509711bf355 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:22:25.878547 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.auth.core [None req-057d78e4-2050-4deb-9d03-c4c071c1bdca None None] MFA Rules not processed for user `d6430593eecf45d8b891c3b55de4a944`. Rule list: `[]` (Enabled: `True`). {{(pid=81676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:22:25.879744 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-057d78e4-2050-4deb-9d03-c4c071c1bdca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:22:25.944103 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1376/2771] 10.222.0.46 () {58 vars in 1009 bytes} [Mon Oct 10 12:22:25 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:22:25.965012 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1396/2772] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:22:25 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:22:25.978885 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057d78e4-2050-4deb-9d03-c4c071c1bdca None None] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:22:25.980251 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-057d78e4-2050-4deb-9d03-c4c071c1bdca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:22:26.040967 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057d78e4-2050-4deb-9d03-c4c071c1bdca None None] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:22:26.044888 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e267bf77-e34a-435c-937a-cb556e474e91 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3ae093470484b93b23dcfae1657df9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad3772f7b6a47c28b7ba37a9316a3c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:22:26.045551 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e267bf77-e34a-435c-937a-cb556e474e91 service cinder] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:22:26.045775 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e267bf77-e34a-435c-937a-cb556e474e91 service cinder] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:22:26.046053 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e267bf77-e34a-435c-937a-cb556e474e91 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:22:26.054797 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e267bf77-e34a-435c-937a-cb556e474e91 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:22:26.057002 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e267bf77-e34a-435c-937a-cb556e474e91 service cinder] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:22:26.066660 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1377/2773] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 10 12:22:25 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:22:26.088807 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805909b2-15a9-4370-be93-4509711bf355 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:22:26.158597 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805909b2-15a9-4370-be93-4509711bf355 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:22:26.166440 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e267bf77-e34a-435c-937a-cb556e474e91 service cinder] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:22:26.167846 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c52c20f-265f-44c4-84d4-aefc03d94e58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:22:26.168259 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-e267bf77-e34a-435c-937a-cb556e474e91 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:22:26.168667 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c52c20f-265f-44c4-84d4-aefc03d94e58 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:22:26.168805 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c52c20f-265f-44c4-84d4-aefc03d94e58 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:22:26.169074 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c52c20f-265f-44c4-84d4-aefc03d94e58 admin admin] PATH_INFO: `/v3/users/17c0750e56bc424fa46b2d2dd3bac023` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:22:26.170223 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c52c20f-265f-44c4-84d4-aefc03d94e58 admin admin] RBAC: Authorizing `identity:delete_user(user_id=17c0750e56bc424fa46b2d2dd3bac023)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:22:26.172575 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c52c20f-265f-44c4-84d4-aefc03d94e58 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:22:26.186115 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:22:26.188905 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:22:26.239780 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e267bf77-e34a-435c-937a-cb556e474e91 service cinder] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:22:26.245746 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4088c54-1620-47b0-8037-c5f72dd910a0 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15ef3d53ea484237adb0b3d64fdc5480', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad3772f7b6a47c28b7ba37a9316a3c1', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:22:26.246350 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4088c54-1620-47b0-8037-c5f72dd910a0 service swift] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:22:26.246628 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4088c54-1620-47b0-8037-c5f72dd910a0 service swift] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:22:26.246933 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4088c54-1620-47b0-8037-c5f72dd910a0 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:22:26.255196 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4088c54-1620-47b0-8037-c5f72dd910a0 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:22:26.256768 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4088c54-1620-47b0-8037-c5f72dd910a0 service swift] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:22:26.264844 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1378/2774] 10.222.0.46 () {60 vars in 1384 bytes} [Mon Oct 10 12:22:26 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Oct 10 12:22:26.413646 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-6c52c20f-265f-44c4-84d4-aefc03d94e58 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '17c0750e56bc424fa46b2d2dd3bac023'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:22:26.423788 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-6c52c20f-265f-44c4-84d4-aefc03d94e58 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '17c0750e56bc424fa46b2d2dd3bac023'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:22:26.424608 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-6c52c20f-265f-44c4-84d4-aefc03d94e58 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '17c0750e56bc424fa46b2d2dd3bac023'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:22:26.530079 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-6c52c20f-265f-44c4-84d4-aefc03d94e58 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '17c0750e56bc424fa46b2d2dd3bac023'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:22:26.544219 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-6c52c20f-265f-44c4-84d4-aefc03d94e58 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '17c0750e56bc424fa46b2d2dd3bac023'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:22:26.554862 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1397/2775] 10.222.0.46 () {58 vars in 1224 bytes} [Mon Oct 10 12:22:26 2022] DELETE /identity/v3/users/17c0750e56bc424fa46b2d2dd3bac023 => generated 0 bytes in 463 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:22:26.563342 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65307f2c-9d72-4b33-8e43-624f7af51058 None None] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:22:26.563342 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65307f2c-9d72-4b33-8e43-624f7af51058 None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:22:26.563342 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65307f2c-9d72-4b33-8e43-624f7af51058 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:22:26.576727 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.auth.core [None req-65307f2c-9d72-4b33-8e43-624f7af51058 None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:22:26.578173 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-65307f2c-9d72-4b33-8e43-624f7af51058 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:22:26.643271 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1379/2776] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:22:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:22:26.653726 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c52c20f-265f-44c4-84d4-aefc03d94e58 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:22:26.655327 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-6c52c20f-265f-44c4-84d4-aefc03d94e58 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:22:26.714957 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c52c20f-265f-44c4-84d4-aefc03d94e58 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:22:26.718638 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91437d9a-0154-46d6-ad7d-7866529c3874 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58b1c7c85302421cba823ace4201861f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad3772f7b6a47c28b7ba37a9316a3c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:22:26.719463 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91437d9a-0154-46d6-ad7d-7866529c3874 service neutron] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:22:26.720043 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91437d9a-0154-46d6-ad7d-7866529c3874 service neutron] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:22:26.720043 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91437d9a-0154-46d6-ad7d-7866529c3874 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:22:26.730400 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91437d9a-0154-46d6-ad7d-7866529c3874 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:22:26.732029 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91437d9a-0154-46d6-ad7d-7866529c3874 service neutron] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:22:26.743598 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1398/2777] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 10 12:22:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:22:27.163160 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65307f2c-9d72-4b33-8e43-624f7af51058 None None] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:22:27.163160 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-65307f2c-9d72-4b33-8e43-624f7af51058 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:22:27.179701 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65307f2c-9d72-4b33-8e43-624f7af51058 None None] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:22:27.187751 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7424532-2f5f-4600-b9c7-1ac07a1e9b2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:22:27.193388 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea53e8ef-33ee-431d-a63e-3eda24cf6598 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:22:27.193953 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea53e8ef-33ee-431d-a63e-3eda24cf6598 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:22:27.194214 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea53e8ef-33ee-431d-a63e-3eda24cf6598 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:22:27.195506 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7424532-2f5f-4600-b9c7-1ac07a1e9b2c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:22:27.195800 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7424532-2f5f-4600-b9c7-1ac07a1e9b2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:22:27.196635 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7424532-2f5f-4600-b9c7-1ac07a1e9b2c admin admin] PATH_INFO: `/v3/projects/759a4fa2a6994b6892030c4d9c46d7c8` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:22:27.198088 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7424532-2f5f-4600-b9c7-1ac07a1e9b2c admin admin] RBAC: Authorizing `identity:delete_project(project_id=759a4fa2a6994b6892030c4d9c46d7c8)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:22:27.200071 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7424532-2f5f-4600-b9c7-1ac07a1e9b2c admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:22:27.214374 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-ea53e8ef-33ee-431d-a63e-3eda24cf6598 None None] MFA Rules not processed for user `d6430593eecf45d8b891c3b55de4a944`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:22:27.215630 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-ea53e8ef-33ee-431d-a63e-3eda24cf6598 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:22:27.274359 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1399/2778] 10.222.0.46 () {58 vars in 967 bytes} [Mon Oct 10 12:22:27 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:22:27.286801 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea53e8ef-33ee-431d-a63e-3eda24cf6598 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:22:27.288327 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-ea53e8ef-33ee-431d-a63e-3eda24cf6598 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:22:27.307178 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-d7424532-2f5f-4600-b9c7-1ac07a1e9b2c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '759a4fa2a6994b6892030c4d9c46d7c8', 'request_id': 'req-d7424532-2f5f-4600-b9c7-1ac07a1e9b2c'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:22:27.314853 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-d7424532-2f5f-4600-b9c7-1ac07a1e9b2c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '759a4fa2a6994b6892030c4d9c46d7c8', 'request_id': 'req-d7424532-2f5f-4600-b9c7-1ac07a1e9b2c'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:22:27.322690 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-d7424532-2f5f-4600-b9c7-1ac07a1e9b2c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '759a4fa2a6994b6892030c4d9c46d7c8', 'request_id': 'req-d7424532-2f5f-4600-b9c7-1ac07a1e9b2c'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:22:27.336306 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:22:27.365971 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea53e8ef-33ee-431d-a63e-3eda24cf6598 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:22:27.410085 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-d7424532-2f5f-4600-b9c7-1ac07a1e9b2c admin admin] The token cache is being invalidate because project 759a4fa2a6994b6892030c4d9c46d7c8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81676) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 10 12:22:27.410513 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-d7424532-2f5f-4600-b9c7-1ac07a1e9b2c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:22:27.411275 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-d7424532-2f5f-4600-b9c7-1ac07a1e9b2c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:22:27.412932 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1380/2779] 10.222.0.46 () {58 vars in 1233 bytes} [Mon Oct 10 12:22:27 2022] DELETE /identity/v3/projects/759a4fa2a6994b6892030c4d9c46d7c8 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:22:27.417804 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb340573-df79-4c16-95ac-63f54a14ad4d None None] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:22:27.418093 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb340573-df79-4c16-95ac-63f54a14ad4d None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:22:27.418374 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb340573-df79-4c16-95ac-63f54a14ad4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:22:27.424824 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54100604-9480-45af-83fb-923a68b086bb service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15ef3d53ea484237adb0b3d64fdc5480', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad3772f7b6a47c28b7ba37a9316a3c1', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:22:27.425501 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54100604-9480-45af-83fb-923a68b086bb service swift] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:22:27.425774 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54100604-9480-45af-83fb-923a68b086bb service swift] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:22:27.426064 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54100604-9480-45af-83fb-923a68b086bb service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:22:27.428696 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-54100604-9480-45af-83fb-923a68b086bb service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:22:27.439114 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.auth.core [None req-eb340573-df79-4c16-95ac-63f54a14ad4d None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:22:27.440265 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-eb340573-df79-4c16-95ac-63f54a14ad4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:22:27.467111 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:22:27.499440 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54100604-9480-45af-83fb-923a68b086bb service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:22:27.501221 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54100604-9480-45af-83fb-923a68b086bb service swift] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:22:27.509349 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1381/2780] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:22:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:22:27.512616 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1400/2781] 10.222.0.46 () {60 vars in 1384 bytes} [Mon Oct 10 12:22:27 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 224 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Oct 10 12:22:27.518253 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b455f9-7a11-4780-9f92-8b4e757732b5 None None] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:22:27.518550 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b455f9-7a11-4780-9f92-8b4e757732b5 None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:22:27.518886 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b455f9-7a11-4780-9f92-8b4e757732b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:22:27.544763 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.auth.core [None req-f0b455f9-7a11-4780-9f92-8b4e757732b5 None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:22:27.546034 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-f0b455f9-7a11-4780-9f92-8b4e757732b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:22:27.555319 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1382/2782] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:22:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:22:27.559642 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54100604-9480-45af-83fb-923a68b086bb service swift] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:22:27.570504 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54100604-9480-45af-83fb-923a68b086bb service swift] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:22:27.575290 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6171f8f3-7581-4832-bf62-69612497219a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:22:27.575893 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6171f8f3-7581-4832-bf62-69612497219a admin admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:22:27.576299 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6171f8f3-7581-4832-bf62-69612497219a admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:22:27.576704 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6171f8f3-7581-4832-bf62-69612497219a admin admin] PATH_INFO: `/v3/domains` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:22:27.577363 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6171f8f3-7581-4832-bf62-69612497219a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 12:22:27.577672 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6171f8f3-7581-4832-bf62-69612497219a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:22:27.579419 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6171f8f3-7581-4832-bf62-69612497219a admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:22:27.591948 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1401/2783] 10.222.0.46 () {58 vars in 1153 bytes} [Mon Oct 10 12:22:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 12:22:27.596448 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b455f9-7a11-4780-9f92-8b4e757732b5 None None] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:22:27.606946 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b455f9-7a11-4780-9f92-8b4e757732b5 None None] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:22:27.611738 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a252bce-bca5-4c51-85f4-6ddb66adec17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:22:27.612356 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a252bce-bca5-4c51-85f4-6ddb66adec17 admin admin] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:22:27.612639 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a252bce-bca5-4c51-85f4-6ddb66adec17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:22:27.612917 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a252bce-bca5-4c51-85f4-6ddb66adec17 admin admin] PATH_INFO: `/v3/projects` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:22:27.613578 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a252bce-bca5-4c51-85f4-6ddb66adec17 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:22:27.615667 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a252bce-bca5-4c51-85f4-6ddb66adec17 admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:22:27.725655 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1383/2784] 10.222.0.46 () {60 vars in 1153 bytes} [Mon Oct 10 12:22:27 2022] POST /identity/v3/projects => generated 375 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 10 12:22:27.730691 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6171f8f3-7581-4832-bf62-69612497219a admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:22:27.740726 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6171f8f3-7581-4832-bf62-69612497219a admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:22:27.798978 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86886b7a-0633-4757-abd4-9f4ec774bf23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:22:27.799544 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86886b7a-0633-4757-abd4-9f4ec774bf23 admin admin] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:22:27.799828 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86886b7a-0633-4757-abd4-9f4ec774bf23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:22:27.800108 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86886b7a-0633-4757-abd4-9f4ec774bf23 admin admin] PATH_INFO: `/v3/users` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:22:27.800736 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86886b7a-0633-4757-abd4-9f4ec774bf23 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:22:27.802700 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86886b7a-0633-4757-abd4-9f4ec774bf23 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:22:27.814589 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:22:27.949676 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1402/2785] 10.222.0.46 () {60 vars in 1144 bytes} [Mon Oct 10 12:22:27 2022] POST /identity/v3/users => generated 288 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 12:22:27.950933 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a252bce-bca5-4c51-85f4-6ddb66adec17 admin admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:22:27.962459 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a252bce-bca5-4c51-85f4-6ddb66adec17 admin admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:22:27.967263 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951dd1d5-331f-46c8-83c1-77f119031377 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:22:27.968022 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951dd1d5-331f-46c8-83c1-77f119031377 admin admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:22:27.968306 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951dd1d5-331f-46c8-83c1-77f119031377 admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:22:27.968584 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951dd1d5-331f-46c8-83c1-77f119031377 admin admin] PATH_INFO: `/v3/roles` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:22:27.969238 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951dd1d5-331f-46c8-83c1-77f119031377 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:22:27.970847 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951dd1d5-331f-46c8-83c1-77f119031377 admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:22:27.986278 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1384/2786] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 10 12:22:27 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 10 12:22:27.987467 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86886b7a-0633-4757-abd4-9f4ec774bf23 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:22:27.998524 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86886b7a-0633-4757-abd4-9f4ec774bf23 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:22:28.005597 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbd955c-d424-4e4d-96ab-8e7c45e62b2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:22:28.006239 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbd955c-d424-4e4d-96ab-8e7c45e62b2f admin admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:22:28.006520 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbd955c-d424-4e4d-96ab-8e7c45e62b2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:22:28.011196 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbd955c-d424-4e4d-96ab-8e7c45e62b2f admin admin] PATH_INFO: `/v3/roles` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:22:28.011196 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bbd955c-d424-4e4d-96ab-8e7c45e62b2f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:22:28.011196 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bbd955c-d424-4e4d-96ab-8e7c45e62b2f admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:22:28.019979 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1403/2787] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 10 12:22:27 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 12:22:28.024757 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951dd1d5-331f-46c8-83c1-77f119031377 admin admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:22:28.035026 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951dd1d5-331f-46c8-83c1-77f119031377 admin admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:22:28.040631 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f9b2a2-86bf-42f4-a130-fbdbd79317b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:22:28.041331 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f9b2a2-86bf-42f4-a130-fbdbd79317b7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:22:28.041614 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f9b2a2-86bf-42f4-a130-fbdbd79317b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:22:28.041893 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f9b2a2-86bf-42f4-a130-fbdbd79317b7 admin admin] PATH_INFO: `/v3/projects/d631ef3b88d440f1819d18da7117ee21/users/d77069477eb54f2aa70e71de2ada760c/roles/b2cb2a6e818e47338ce071e6fedfc673` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:22:28.062796 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f9b2a2-86bf-42f4-a130-fbdbd79317b7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d631ef3b88d440f1819d18da7117ee21, user_id=d77069477eb54f2aa70e71de2ada760c, role_id=b2cb2a6e818e47338ce071e6fedfc673)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:22:28.062796 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f9b2a2-86bf-42f4-a130-fbdbd79317b7 admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:22:28.189865 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1385/2788] 10.222.0.46 () {60 vars in 1483 bytes} [Mon Oct 10 12:22:28 2022] PUT /identity/v3/projects/d631ef3b88d440f1819d18da7117ee21/users/d77069477eb54f2aa70e71de2ada760c/roles/b2cb2a6e818e47338ce071e6fedfc673 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:22:28.200830 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286b96f3-4d89-4408-a87d-7575bc043bf9 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:22:28.201281 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286b96f3-4d89-4408-a87d-7575bc043bf9 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:22:28.201399 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286b96f3-4d89-4408-a87d-7575bc043bf9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:22:28.232317 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-286b96f3-4d89-4408-a87d-7575bc043bf9 None None] MFA Rules not processed for user `d77069477eb54f2aa70e71de2ada760c`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:22:28.233550 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-286b96f3-4d89-4408-a87d-7575bc043bf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:22:28.292302 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683d3750-2f94-4495-9fb7-abc290b7969f None None] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:22:28.292571 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683d3750-2f94-4495-9fb7-abc290b7969f None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:22:28.292850 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683d3750-2f94-4495-9fb7-abc290b7969f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:22:28.314141 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.auth.core [None req-683d3750-2f94-4495-9fb7-abc290b7969f None None] MFA Rules not processed for user `d6430593eecf45d8b891c3b55de4a944`. Rule list: `[]` (Enabled: `True`). {{(pid=81676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:22:28.315298 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-683d3750-2f94-4495-9fb7-abc290b7969f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:22:28.315447 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1404/2789] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:22:28 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:22:28.328217 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286b96f3-4d89-4408-a87d-7575bc043bf9 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:22:28.329620 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-286b96f3-4d89-4408-a87d-7575bc043bf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:22:28.379494 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1386/2790] 10.222.0.46 () {58 vars in 1009 bytes} [Mon Oct 10 12:22:28 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:22:28.390145 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286b96f3-4d89-4408-a87d-7575bc043bf9 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:22:28.392590 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683d3750-2f94-4495-9fb7-abc290b7969f None None] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:22:28.393917 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-683d3750-2f94-4495-9fb7-abc290b7969f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:22:28.394049 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a97c037-eb4e-42d8-ba9d-7f35fc6a5558 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3ae093470484b93b23dcfae1657df9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad3772f7b6a47c28b7ba37a9316a3c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:22:28.394651 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a97c037-eb4e-42d8-ba9d-7f35fc6a5558 service cinder] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:22:28.394974 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a97c037-eb4e-42d8-ba9d-7f35fc6a5558 service cinder] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:22:28.395252 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a97c037-eb4e-42d8-ba9d-7f35fc6a5558 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:22:28.405567 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a97c037-eb4e-42d8-ba9d-7f35fc6a5558 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:22:28.407038 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a97c037-eb4e-42d8-ba9d-7f35fc6a5558 service cinder] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:22:28.417605 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1405/2791] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 10 12:22:28 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 10 12:22:28.474625 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683d3750-2f94-4495-9fb7-abc290b7969f None None] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:22:28.479786 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca98879-71cc-44df-9a71-2c9379586449 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15ef3d53ea484237adb0b3d64fdc5480', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad3772f7b6a47c28b7ba37a9316a3c1', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:22:28.480400 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca98879-71cc-44df-9a71-2c9379586449 service swift] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:22:28.480678 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca98879-71cc-44df-9a71-2c9379586449 service swift] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:22:28.480949 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca98879-71cc-44df-9a71-2c9379586449 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:22:28.489749 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca98879-71cc-44df-9a71-2c9379586449 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:22:28.491374 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca98879-71cc-44df-9a71-2c9379586449 service swift] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:22:28.496310 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:22:28.500747 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1387/2792] 10.222.0.46 () {60 vars in 1384 bytes} [Mon Oct 10 12:22:28 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Oct 10 12:22:29.200963 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3789dace-914e-4c1a-91c9-ea63c7721b25 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:22:29.201452 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3789dace-914e-4c1a-91c9-ea63c7721b25 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:22:29.201576 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3789dace-914e-4c1a-91c9-ea63c7721b25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:22:29.222861 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-3789dace-914e-4c1a-91c9-ea63c7721b25 None None] MFA Rules not processed for user `d6430593eecf45d8b891c3b55de4a944`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:22:29.222861 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-3789dace-914e-4c1a-91c9-ea63c7721b25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:22:29.235710 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1406/2793] 10.222.0.46 () {58 vars in 967 bytes} [Mon Oct 10 12:22:29 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:22:29.249071 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca98879-71cc-44df-9a71-2c9379586449 service swift] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:22:29.259953 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca98879-71cc-44df-9a71-2c9379586449 service swift] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:22:29.265163 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cd6db8-b48b-4218-be37-99d3898885f7 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15ef3d53ea484237adb0b3d64fdc5480', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad3772f7b6a47c28b7ba37a9316a3c1', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:22:29.265777 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cd6db8-b48b-4218-be37-99d3898885f7 service swift] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:22:29.266059 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cd6db8-b48b-4218-be37-99d3898885f7 service swift] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:22:29.266356 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cd6db8-b48b-4218-be37-99d3898885f7 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:22:29.277832 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34cd6db8-b48b-4218-be37-99d3898885f7 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:22:29.279568 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34cd6db8-b48b-4218-be37-99d3898885f7 service swift] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:22:29.289194 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1388/2794] 10.222.0.46 () {60 vars in 1384 bytes} [Mon Oct 10 12:22:29 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 10 12:22:29.472408 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3789dace-914e-4c1a-91c9-ea63c7721b25 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:22:29.473814 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-3789dace-914e-4c1a-91c9-ea63c7721b25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:22:29.542316 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3789dace-914e-4c1a-91c9-ea63c7721b25 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:22:29.547447 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86dd2b80-7080-4d43-9a99-cfa755a4187d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:22:29.548082 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dd2b80-7080-4d43-9a99-cfa755a4187d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:22:29.548363 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dd2b80-7080-4d43-9a99-cfa755a4187d admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:22:29.548638 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dd2b80-7080-4d43-9a99-cfa755a4187d admin admin] PATH_INFO: `/v3/users/b4f1817a080049429541a5c5e3929cf8` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:22:29.549870 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86dd2b80-7080-4d43-9a99-cfa755a4187d admin admin] RBAC: Authorizing `identity:delete_user(user_id=b4f1817a080049429541a5c5e3929cf8)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:22:29.552016 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86dd2b80-7080-4d43-9a99-cfa755a4187d admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:22:29.567714 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:22:29.857830 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-86dd2b80-7080-4d43-9a99-cfa755a4187d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b4f1817a080049429541a5c5e3929cf8'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:22:29.867534 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-86dd2b80-7080-4d43-9a99-cfa755a4187d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b4f1817a080049429541a5c5e3929cf8'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:22:29.868359 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-86dd2b80-7080-4d43-9a99-cfa755a4187d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b4f1817a080049429541a5c5e3929cf8'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:22:29.941020 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-86dd2b80-7080-4d43-9a99-cfa755a4187d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b4f1817a080049429541a5c5e3929cf8'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:22:29.959463 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-86dd2b80-7080-4d43-9a99-cfa755a4187d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b4f1817a080049429541a5c5e3929cf8'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:22:29.966767 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1407/2795] 10.222.0.46 () {58 vars in 1224 bytes} [Mon Oct 10 12:22:29 2022] DELETE /identity/v3/users/b4f1817a080049429541a5c5e3929cf8 => generated 0 bytes in 495 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 10 12:22:29.971299 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9284e4f2-8dd0-46a2-84a5-164413d2451b None None] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:22:29.971590 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9284e4f2-8dd0-46a2-84a5-164413d2451b None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:22:29.971867 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9284e4f2-8dd0-46a2-84a5-164413d2451b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:22:29.991904 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.auth.core [None req-9284e4f2-8dd0-46a2-84a5-164413d2451b None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:22:29.995757 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-9284e4f2-8dd0-46a2-84a5-164413d2451b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:22:30.076183 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1389/2796] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:22:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:22:30.084733 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86dd2b80-7080-4d43-9a99-cfa755a4187d admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:22:30.089080 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-86dd2b80-7080-4d43-9a99-cfa755a4187d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:22:30.154065 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86dd2b80-7080-4d43-9a99-cfa755a4187d admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:22:30.157658 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0356f69-45dc-4234-86f2-bd4e7e9e0c4e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58b1c7c85302421cba823ace4201861f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad3772f7b6a47c28b7ba37a9316a3c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:22:30.158270 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0356f69-45dc-4234-86f2-bd4e7e9e0c4e service neutron] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:22:30.158544 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0356f69-45dc-4234-86f2-bd4e7e9e0c4e service neutron] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:22:30.158838 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0356f69-45dc-4234-86f2-bd4e7e9e0c4e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:22:30.170063 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0356f69-45dc-4234-86f2-bd4e7e9e0c4e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:22:30.171789 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0356f69-45dc-4234-86f2-bd4e7e9e0c4e service neutron] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:22:30.189909 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1408/2797] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 10 12:22:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:22:30.692007 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9284e4f2-8dd0-46a2-84a5-164413d2451b None None] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:22:30.693905 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-9284e4f2-8dd0-46a2-84a5-164413d2451b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:22:30.707132 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9284e4f2-8dd0-46a2-84a5-164413d2451b None None] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:22:30.712812 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea964573-e8ae-46f7-8f3a-aa6b99f57588 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:22:30.713530 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea964573-e8ae-46f7-8f3a-aa6b99f57588 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:22:30.713816 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea964573-e8ae-46f7-8f3a-aa6b99f57588 admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:22:30.714095 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea964573-e8ae-46f7-8f3a-aa6b99f57588 admin admin] PATH_INFO: `/v3/projects/999aa6678db8480d876992e96a9d577f` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:22:30.715295 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea964573-e8ae-46f7-8f3a-aa6b99f57588 admin admin] RBAC: Authorizing `identity:delete_project(project_id=999aa6678db8480d876992e96a9d577f)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:22:30.717228 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea964573-e8ae-46f7-8f3a-aa6b99f57588 admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:22:30.864996 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-ea964573-e8ae-46f7-8f3a-aa6b99f57588 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '999aa6678db8480d876992e96a9d577f', 'request_id': 'req-ea964573-e8ae-46f7-8f3a-aa6b99f57588'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:22:30.871422 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-ea964573-e8ae-46f7-8f3a-aa6b99f57588 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '999aa6678db8480d876992e96a9d577f', 'request_id': 'req-ea964573-e8ae-46f7-8f3a-aa6b99f57588'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:22:30.878288 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-ea964573-e8ae-46f7-8f3a-aa6b99f57588 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '999aa6678db8480d876992e96a9d577f', 'request_id': 'req-ea964573-e8ae-46f7-8f3a-aa6b99f57588'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:22:31.031259 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-ea964573-e8ae-46f7-8f3a-aa6b99f57588 admin admin] The token cache is being invalidate because project 999aa6678db8480d876992e96a9d577f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81676) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 10 12:22:31.031558 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-ea964573-e8ae-46f7-8f3a-aa6b99f57588 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:22:31.032205 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-ea964573-e8ae-46f7-8f3a-aa6b99f57588 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:22:31.033736 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1390/2798] 10.222.0.46 () {58 vars in 1233 bytes} [Mon Oct 10 12:22:30 2022] DELETE /identity/v3/projects/999aa6678db8480d876992e96a9d577f => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:22:31.037735 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0356f69-45dc-4234-86f2-bd4e7e9e0c4e service neutron] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:22:31.047974 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0356f69-45dc-4234-86f2-bd4e7e9e0c4e service neutron] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:22:31.107826 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5aed11-8b62-411e-9d69-19cd070df222 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:22:31.108485 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5aed11-8b62-411e-9d69-19cd070df222 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:22:31.108766 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5aed11-8b62-411e-9d69-19cd070df222 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:22:31.109041 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5aed11-8b62-411e-9d69-19cd070df222 admin admin] PATH_INFO: `/v3/users/cc77924c7f234cc2be9544a607058621` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:22:31.110263 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5aed11-8b62-411e-9d69-19cd070df222 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cc77924c7f234cc2be9544a607058621)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:22:31.112255 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5aed11-8b62-411e-9d69-19cd070df222 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:22:31.324408 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:22:31.333616 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-ee5aed11-8b62-411e-9d69-19cd070df222 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cc77924c7f234cc2be9544a607058621'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:22:31.341986 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-ee5aed11-8b62-411e-9d69-19cd070df222 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cc77924c7f234cc2be9544a607058621'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:22:31.343005 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-ee5aed11-8b62-411e-9d69-19cd070df222 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cc77924c7f234cc2be9544a607058621'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:22:31.434972 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-ee5aed11-8b62-411e-9d69-19cd070df222 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cc77924c7f234cc2be9544a607058621'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:22:31.448827 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-ee5aed11-8b62-411e-9d69-19cd070df222 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cc77924c7f234cc2be9544a607058621'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:22:31.456512 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1409/2799] 10.222.0.46 () {58 vars in 1224 bytes} [Mon Oct 10 12:22:31 2022] DELETE /identity/v3/users/cc77924c7f234cc2be9544a607058621 => generated 0 bytes in 419 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:22:31.928669 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea964573-e8ae-46f7-8f3a-aa6b99f57588 admin admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:22:31.930138 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-ea964573-e8ae-46f7-8f3a-aa6b99f57588 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:22:31.944281 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea964573-e8ae-46f7-8f3a-aa6b99f57588 admin admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:22:32.004811 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f52fc2-b35c-4afb-9e14-ac4c0558d666 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:22:32.005444 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f52fc2-b35c-4afb-9e14-ac4c0558d666 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:22:32.005720 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f52fc2-b35c-4afb-9e14-ac4c0558d666 admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:22:32.005993 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f52fc2-b35c-4afb-9e14-ac4c0558d666 admin admin] PATH_INFO: `/v3/projects/b5b324c81c7a482bbb61caae5eed59ce` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:22:32.007133 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f52fc2-b35c-4afb-9e14-ac4c0558d666 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b5b324c81c7a482bbb61caae5eed59ce)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:22:32.009039 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f52fc2-b35c-4afb-9e14-ac4c0558d666 admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:22:32.020330 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:22:32.096412 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-d7f52fc2-b35c-4afb-9e14-ac4c0558d666 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b5b324c81c7a482bbb61caae5eed59ce', 'request_id': 'req-d7f52fc2-b35c-4afb-9e14-ac4c0558d666'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:22:32.102003 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-d7f52fc2-b35c-4afb-9e14-ac4c0558d666 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b5b324c81c7a482bbb61caae5eed59ce', 'request_id': 'req-d7f52fc2-b35c-4afb-9e14-ac4c0558d666'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:22:32.108665 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-d7f52fc2-b35c-4afb-9e14-ac4c0558d666 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b5b324c81c7a482bbb61caae5eed59ce', 'request_id': 'req-d7f52fc2-b35c-4afb-9e14-ac4c0558d666'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:22:32.175620 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-d7f52fc2-b35c-4afb-9e14-ac4c0558d666 admin admin] The token cache is being invalidate because project b5b324c81c7a482bbb61caae5eed59ce was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81676) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 10 12:22:32.175910 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-d7f52fc2-b35c-4afb-9e14-ac4c0558d666 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:22:32.176544 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-d7f52fc2-b35c-4afb-9e14-ac4c0558d666 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:22:32.178050 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1391/2800] 10.222.0.46 () {58 vars in 1233 bytes} [Mon Oct 10 12:22:31 2022] DELETE /identity/v3/projects/b5b324c81c7a482bbb61caae5eed59ce => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:22:32.183528 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e27cc99-114f-46b9-b7c8-d5e71f786312 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:22:32.184021 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e27cc99-114f-46b9-b7c8-d5e71f786312 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:22:32.184126 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e27cc99-114f-46b9-b7c8-d5e71f786312 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:22:32.204321 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-7e27cc99-114f-46b9-b7c8-d5e71f786312 None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:22:32.205515 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-7e27cc99-114f-46b9-b7c8-d5e71f786312 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:22:32.263789 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1410/2801] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:22:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:22:32.271610 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51a7b88-3b91-4264-bd37-c4d5aa56fef0 None None] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:22:32.272118 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51a7b88-3b91-4264-bd37-c4d5aa56fef0 None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:22:32.272684 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51a7b88-3b91-4264-bd37-c4d5aa56fef0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:22:32.293464 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.auth.core [None req-c51a7b88-3b91-4264-bd37-c4d5aa56fef0 None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:22:32.295376 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-c51a7b88-3b91-4264-bd37-c4d5aa56fef0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:22:32.303478 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1392/2802] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:22:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:22:32.307423 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e27cc99-114f-46b9-b7c8-d5e71f786312 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:22:32.317263 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e27cc99-114f-46b9-b7c8-d5e71f786312 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:22:32.321836 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f553c72d-873f-4537-b5a6-42ac75e0b42a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:22:32.322446 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f553c72d-873f-4537-b5a6-42ac75e0b42a admin admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:22:32.322758 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f553c72d-873f-4537-b5a6-42ac75e0b42a admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:22:32.323038 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f553c72d-873f-4537-b5a6-42ac75e0b42a admin admin] PATH_INFO: `/v3/domains` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:22:32.323681 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f553c72d-873f-4537-b5a6-42ac75e0b42a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 12:22:32.323986 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f553c72d-873f-4537-b5a6-42ac75e0b42a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:22:32.325625 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f553c72d-873f-4537-b5a6-42ac75e0b42a admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:22:32.342059 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1411/2803] 10.222.0.46 () {58 vars in 1153 bytes} [Mon Oct 10 12:22:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 12:22:32.344099 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51a7b88-3b91-4264-bd37-c4d5aa56fef0 None None] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:22:32.353980 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51a7b88-3b91-4264-bd37-c4d5aa56fef0 None None] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:22:32.359296 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81acbdfb-23e7-41ab-ae7f-83c8c5f1e1a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:22:32.359904 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81acbdfb-23e7-41ab-ae7f-83c8c5f1e1a0 admin admin] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:22:32.360188 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81acbdfb-23e7-41ab-ae7f-83c8c5f1e1a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:22:32.360495 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81acbdfb-23e7-41ab-ae7f-83c8c5f1e1a0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:22:32.361173 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81acbdfb-23e7-41ab-ae7f-83c8c5f1e1a0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:22:32.363158 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81acbdfb-23e7-41ab-ae7f-83c8c5f1e1a0 admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:22:32.454403 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1393/2804] 10.222.0.46 () {60 vars in 1153 bytes} [Mon Oct 10 12:22:32 2022] POST /identity/v3/projects => generated 369 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 12:22:32.458703 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f553c72d-873f-4537-b5a6-42ac75e0b42a admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:22:32.473318 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f553c72d-873f-4537-b5a6-42ac75e0b42a admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:22:32.532944 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3efc15a-c584-442c-bb7a-fe3fdc9faa08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:22:32.533572 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3efc15a-c584-442c-bb7a-fe3fdc9faa08 admin admin] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:22:32.533853 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3efc15a-c584-442c-bb7a-fe3fdc9faa08 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:22:32.534129 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3efc15a-c584-442c-bb7a-fe3fdc9faa08 admin admin] PATH_INFO: `/v3/users` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:22:32.534889 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3efc15a-c584-442c-bb7a-fe3fdc9faa08 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:22:32.536763 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3efc15a-c584-442c-bb7a-fe3fdc9faa08 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:22:32.661666 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1412/2805] 10.222.0.46 () {60 vars in 1144 bytes} [Mon Oct 10 12:22:32 2022] POST /identity/v3/users => generated 285 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 10 12:22:32.665905 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81acbdfb-23e7-41ab-ae7f-83c8c5f1e1a0 admin admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:22:32.676096 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81acbdfb-23e7-41ab-ae7f-83c8c5f1e1a0 admin admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:22:32.680530 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6d6474-b501-48be-a2f3-5634e0b8714a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:22:32.681117 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6d6474-b501-48be-a2f3-5634e0b8714a admin admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:22:32.681393 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6d6474-b501-48be-a2f3-5634e0b8714a admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:22:32.681665 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6d6474-b501-48be-a2f3-5634e0b8714a admin admin] PATH_INFO: `/v3/roles` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:22:32.682300 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6d6474-b501-48be-a2f3-5634e0b8714a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:22:32.683936 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6d6474-b501-48be-a2f3-5634e0b8714a admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:22:32.693680 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1394/2806] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 10 12:22:32 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 12:22:32.697770 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3efc15a-c584-442c-bb7a-fe3fdc9faa08 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:22:32.708820 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3efc15a-c584-442c-bb7a-fe3fdc9faa08 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:22:32.713836 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d7d4b4-b69b-44e2-9bfd-bda6f0ea9852 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:22:32.714435 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d7d4b4-b69b-44e2-9bfd-bda6f0ea9852 admin admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:22:32.714743 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d7d4b4-b69b-44e2-9bfd-bda6f0ea9852 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:22:32.715022 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d7d4b4-b69b-44e2-9bfd-bda6f0ea9852 admin admin] PATH_INFO: `/v3/roles` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:22:32.716279 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d7d4b4-b69b-44e2-9bfd-bda6f0ea9852 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:22:32.717924 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d7d4b4-b69b-44e2-9bfd-bda6f0ea9852 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:22:32.728125 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1413/2807] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 10 12:22:32 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 12:22:32.732565 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6d6474-b501-48be-a2f3-5634e0b8714a admin admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:22:32.742429 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6d6474-b501-48be-a2f3-5634e0b8714a admin admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:22:32.746930 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f443fdc1-96e4-4804-a686-8c0e3132a5ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:22:32.747577 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f443fdc1-96e4-4804-a686-8c0e3132a5ab admin admin] REQUEST_METHOD: `PUT` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:22:32.747859 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f443fdc1-96e4-4804-a686-8c0e3132a5ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:22:32.748138 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f443fdc1-96e4-4804-a686-8c0e3132a5ab admin admin] PATH_INFO: `/v3/projects/2406e23f06df493ab9afcba4b0567de1/users/2df714a3944a471fbecf2ac37e7e7e45/roles/b2cb2a6e818e47338ce071e6fedfc673` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:22:32.763165 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f443fdc1-96e4-4804-a686-8c0e3132a5ab admin admin] RBAC: Authorizing `identity:create_grant(project_id=2406e23f06df493ab9afcba4b0567de1, user_id=2df714a3944a471fbecf2ac37e7e7e45, role_id=b2cb2a6e818e47338ce071e6fedfc673)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:22:32.765787 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f443fdc1-96e4-4804-a686-8c0e3132a5ab admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:22:32.823243 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1395/2808] 10.222.0.46 () {60 vars in 1483 bytes} [Mon Oct 10 12:22:32 2022] PUT /identity/v3/projects/2406e23f06df493ab9afcba4b0567de1/users/2df714a3944a471fbecf2ac37e7e7e45/roles/b2cb2a6e818e47338ce071e6fedfc673 => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 10 12:22:32.838193 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb502da-25e5-40a2-98a9-7e8b9f5becfc None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:22:32.838470 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb502da-25e5-40a2-98a9-7e8b9f5becfc None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:22:32.838785 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb502da-25e5-40a2-98a9-7e8b9f5becfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:22:32.868883 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-dcb502da-25e5-40a2-98a9-7e8b9f5becfc None None] MFA Rules not processed for user `2df714a3944a471fbecf2ac37e7e7e45`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:22:32.870132 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-dcb502da-25e5-40a2-98a9-7e8b9f5becfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:22:32.876177 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:22:32.937799 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1414/2809] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:22:32 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:22:32.942308 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f443fdc1-96e4-4804-a686-8c0e3132a5ab admin admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:22:33.008301 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f443fdc1-96e4-4804-a686-8c0e3132a5ab admin admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:22:33.013797 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcadbcd-24f4-437c-b606-23876b23814a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:22:33.014409 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcadbcd-24f4-437c-b606-23876b23814a admin admin] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:22:33.014722 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcadbcd-24f4-437c-b606-23876b23814a admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:22:33.015004 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcadbcd-24f4-437c-b606-23876b23814a admin admin] PATH_INFO: `/v3/projects` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:22:33.015654 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fcadbcd-24f4-437c-b606-23876b23814a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:22:33.017563 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fcadbcd-24f4-437c-b606-23876b23814a admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:22:33.096693 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1396/2810] 10.222.0.46 () {60 vars in 1153 bytes} [Mon Oct 10 12:22:32 2022] POST /identity/v3/projects => generated 369 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 10 12:22:33.100801 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb502da-25e5-40a2-98a9-7e8b9f5becfc None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:22:33.111732 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb502da-25e5-40a2-98a9-7e8b9f5becfc None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:22:33.167556 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06ce607-fca6-4e38-aed9-4198b4881a88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:22:33.168170 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06ce607-fca6-4e38-aed9-4198b4881a88 admin admin] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:22:33.168448 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06ce607-fca6-4e38-aed9-4198b4881a88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:22:33.168767 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06ce607-fca6-4e38-aed9-4198b4881a88 admin admin] PATH_INFO: `/v3/users` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:22:33.169400 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06ce607-fca6-4e38-aed9-4198b4881a88 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:22:33.171424 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06ce607-fca6-4e38-aed9-4198b4881a88 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:22:33.255532 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1415/2811] 10.222.0.46 () {60 vars in 1144 bytes} [Mon Oct 10 12:22:33 2022] POST /identity/v3/users => generated 291 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 12:22:33.259407 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcadbcd-24f4-437c-b606-23876b23814a admin admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:22:33.270586 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcadbcd-24f4-437c-b606-23876b23814a admin admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:22:33.275416 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f930655a-7a87-4ea8-b679-1565516ebe23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:22:33.276023 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f930655a-7a87-4ea8-b679-1565516ebe23 admin admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:22:33.276304 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f930655a-7a87-4ea8-b679-1565516ebe23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:22:33.276596 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f930655a-7a87-4ea8-b679-1565516ebe23 admin admin] PATH_INFO: `/v3/roles` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:22:33.277267 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f930655a-7a87-4ea8-b679-1565516ebe23 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:22:33.278953 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f930655a-7a87-4ea8-b679-1565516ebe23 admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:22:33.289638 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1397/2812] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 10 12:22:33 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 10 12:22:33.293839 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06ce607-fca6-4e38-aed9-4198b4881a88 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:22:33.304803 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06ce607-fca6-4e38-aed9-4198b4881a88 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:22:33.310067 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c187da-b592-4078-a62f-1578f49399c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:22:33.310809 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c187da-b592-4078-a62f-1578f49399c4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:22:33.311093 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c187da-b592-4078-a62f-1578f49399c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:22:33.311370 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c187da-b592-4078-a62f-1578f49399c4 admin admin] PATH_INFO: `/v3/projects/840679fe62464eb8a84ef84970111716/users/5a858c7a10b948e78600099de4a02ed1/roles/0d822a0674b6495fac623877a3b38025` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:22:33.326821 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c187da-b592-4078-a62f-1578f49399c4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=840679fe62464eb8a84ef84970111716, user_id=5a858c7a10b948e78600099de4a02ed1, role_id=0d822a0674b6495fac623877a3b38025)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:22:33.329494 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c187da-b592-4078-a62f-1578f49399c4 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:22:33.439485 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1416/2813] 10.222.0.46 () {60 vars in 1483 bytes} [Mon Oct 10 12:22:33 2022] PUT /identity/v3/projects/840679fe62464eb8a84ef84970111716/users/5a858c7a10b948e78600099de4a02ed1/roles/0d822a0674b6495fac623877a3b38025 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:22:33.443654 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f930655a-7a87-4ea8-b679-1565516ebe23 admin admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:22:33.457412 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f930655a-7a87-4ea8-b679-1565516ebe23 admin admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:22:33.467113 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:22:33.518995 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101bbcf6-9bb5-4230-9f16-c5cfa3b457c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:22:33.519606 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101bbcf6-9bb5-4230-9f16-c5cfa3b457c1 admin admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:22:33.519886 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101bbcf6-9bb5-4230-9f16-c5cfa3b457c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:22:33.520159 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101bbcf6-9bb5-4230-9f16-c5cfa3b457c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:22:33.520805 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101bbcf6-9bb5-4230-9f16-c5cfa3b457c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:22:33.522404 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101bbcf6-9bb5-4230-9f16-c5cfa3b457c1 admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:22:33.533157 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1398/2814] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 10 12:22:33 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 10 12:22:33.540482 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c187da-b592-4078-a62f-1578f49399c4 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:22:33.548985 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c187da-b592-4078-a62f-1578f49399c4 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:22:33.553738 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be9729c-3d96-4055-8a2b-96f6c9b8d3e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:22:33.554403 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be9729c-3d96-4055-8a2b-96f6c9b8d3e4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:22:33.554717 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be9729c-3d96-4055-8a2b-96f6c9b8d3e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:22:33.555002 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be9729c-3d96-4055-8a2b-96f6c9b8d3e4 admin admin] PATH_INFO: `/v3/projects/840679fe62464eb8a84ef84970111716/users/5a858c7a10b948e78600099de4a02ed1/roles/0d822a0674b6495fac623877a3b38025` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:22:33.557023 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be9729c-3d96-4055-8a2b-96f6c9b8d3e4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=840679fe62464eb8a84ef84970111716, user_id=5a858c7a10b948e78600099de4a02ed1, role_id=0d822a0674b6495fac623877a3b38025)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:22:33.560341 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be9729c-3d96-4055-8a2b-96f6c9b8d3e4 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:22:33.570819 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1417/2815] 10.222.0.46 () {60 vars in 1483 bytes} [Mon Oct 10 12:22:33 2022] PUT /identity/v3/projects/840679fe62464eb8a84ef84970111716/users/5a858c7a10b948e78600099de4a02ed1/roles/0d822a0674b6495fac623877a3b38025 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:22:33.581248 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4006600c-fbda-4f96-bf41-a706281b58b8 None None] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:22:33.581557 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4006600c-fbda-4f96-bf41-a706281b58b8 None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:22:33.581832 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4006600c-fbda-4f96-bf41-a706281b58b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:22:33.610691 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.auth.core [None req-4006600c-fbda-4f96-bf41-a706281b58b8 None None] MFA Rules not processed for user `5a858c7a10b948e78600099de4a02ed1`. Rule list: `[]` (Enabled: `True`). {{(pid=81676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:22:33.612709 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-4006600c-fbda-4f96-bf41-a706281b58b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:22:33.682366 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1399/2816] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:22:33 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:22:33.693773 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be9729c-3d96-4055-8a2b-96f6c9b8d3e4 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:22:33.695544 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-3be9729c-3d96-4055-8a2b-96f6c9b8d3e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:22:33.753621 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be9729c-3d96-4055-8a2b-96f6c9b8d3e4 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:22:33.757337 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86a8c29-ceed-428e-b8c6-214077f08946 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3ae093470484b93b23dcfae1657df9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad3772f7b6a47c28b7ba37a9316a3c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:22:33.757956 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86a8c29-ceed-428e-b8c6-214077f08946 service cinder] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:22:33.758230 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86a8c29-ceed-428e-b8c6-214077f08946 service cinder] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:22:33.758502 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86a8c29-ceed-428e-b8c6-214077f08946 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:22:33.768026 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86a8c29-ceed-428e-b8c6-214077f08946 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:22:33.769491 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86a8c29-ceed-428e-b8c6-214077f08946 service cinder] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:22:33.779536 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1418/2817] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 10 12:22:33 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:22:35.490128 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4006600c-fbda-4f96-bf41-a706281b58b8 None None] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:22:35.491898 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-4006600c-fbda-4f96-bf41-a706281b58b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:22:35.564119 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4006600c-fbda-4f96-bf41-a706281b58b8 None None] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:22:35.569077 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6445601f-b4a9-4786-a839-2c34ce8e1ec2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:22:35.569767 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6445601f-b4a9-4786-a839-2c34ce8e1ec2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:22:35.574951 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6445601f-b4a9-4786-a839-2c34ce8e1ec2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:22:35.574951 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6445601f-b4a9-4786-a839-2c34ce8e1ec2 admin admin] PATH_INFO: `/v3/users/edddd32b0093470482df3491ed69ad6c` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:22:35.574951 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6445601f-b4a9-4786-a839-2c34ce8e1ec2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=edddd32b0093470482df3491ed69ad6c)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:22:35.574951 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6445601f-b4a9-4786-a839-2c34ce8e1ec2 admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:22:35.785357 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:22:35.796720 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-6445601f-b4a9-4786-a839-2c34ce8e1ec2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'edddd32b0093470482df3491ed69ad6c'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:22:35.802335 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-6445601f-b4a9-4786-a839-2c34ce8e1ec2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'edddd32b0093470482df3491ed69ad6c'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:22:35.803086 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-6445601f-b4a9-4786-a839-2c34ce8e1ec2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'edddd32b0093470482df3491ed69ad6c'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:22:35.901784 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-6445601f-b4a9-4786-a839-2c34ce8e1ec2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'edddd32b0093470482df3491ed69ad6c'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:22:35.916174 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-6445601f-b4a9-4786-a839-2c34ce8e1ec2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'edddd32b0093470482df3491ed69ad6c'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:22:35.923463 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1400/2818] 10.222.0.46 () {58 vars in 1224 bytes} [Mon Oct 10 12:22:35 2022] DELETE /identity/v3/users/edddd32b0093470482df3491ed69ad6c => generated 0 bytes in 434 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:22:35.928001 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1faace40-38ba-4bc2-896c-4b34d5dbe20b None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:22:35.928536 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1faace40-38ba-4bc2-896c-4b34d5dbe20b None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:22:35.928536 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1faace40-38ba-4bc2-896c-4b34d5dbe20b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:22:35.936018 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:22:35.948765 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-1faace40-38ba-4bc2-896c-4b34d5dbe20b None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:22:35.950475 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-1faace40-38ba-4bc2-896c-4b34d5dbe20b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:22:36.011509 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1419/2819] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:22:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 10 12:22:36.021907 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6445601f-b4a9-4786-a839-2c34ce8e1ec2 admin admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:22:36.023358 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-6445601f-b4a9-4786-a839-2c34ce8e1ec2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:22:36.090110 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6445601f-b4a9-4786-a839-2c34ce8e1ec2 admin admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:22:36.092313 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e726ac-8792-4e3a-ad6e-7524c24e096d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58b1c7c85302421cba823ace4201861f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad3772f7b6a47c28b7ba37a9316a3c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:22:36.093003 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e726ac-8792-4e3a-ad6e-7524c24e096d service neutron] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:22:36.093352 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e726ac-8792-4e3a-ad6e-7524c24e096d service neutron] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:22:36.093712 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e726ac-8792-4e3a-ad6e-7524c24e096d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:22:36.103848 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e726ac-8792-4e3a-ad6e-7524c24e096d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:22:36.105288 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e726ac-8792-4e3a-ad6e-7524c24e096d service neutron] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:22:36.116145 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1401/2820] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 10 12:22:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:22:36.428293 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1faace40-38ba-4bc2-896c-4b34d5dbe20b None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:22:36.429732 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-1faace40-38ba-4bc2-896c-4b34d5dbe20b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:22:36.444179 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1faace40-38ba-4bc2-896c-4b34d5dbe20b None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:22:36.450242 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657176b4-cb84-471d-af2e-43a4332d1742 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:22:36.451390 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657176b4-cb84-471d-af2e-43a4332d1742 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:22:36.451978 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657176b4-cb84-471d-af2e-43a4332d1742 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:22:36.452389 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657176b4-cb84-471d-af2e-43a4332d1742 admin admin] PATH_INFO: `/v3/projects/931d44e7d8c84dad9aa3c64eafd899d8` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:22:36.454229 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657176b4-cb84-471d-af2e-43a4332d1742 admin admin] RBAC: Authorizing `identity:delete_project(project_id=931d44e7d8c84dad9aa3c64eafd899d8)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:22:36.459377 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657176b4-cb84-471d-af2e-43a4332d1742 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:22:36.636188 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-657176b4-cb84-471d-af2e-43a4332d1742 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '931d44e7d8c84dad9aa3c64eafd899d8', 'request_id': 'req-657176b4-cb84-471d-af2e-43a4332d1742'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:22:36.641931 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-657176b4-cb84-471d-af2e-43a4332d1742 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '931d44e7d8c84dad9aa3c64eafd899d8', 'request_id': 'req-657176b4-cb84-471d-af2e-43a4332d1742'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:22:36.648115 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-657176b4-cb84-471d-af2e-43a4332d1742 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '931d44e7d8c84dad9aa3c64eafd899d8', 'request_id': 'req-657176b4-cb84-471d-af2e-43a4332d1742'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:22:36.743695 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-657176b4-cb84-471d-af2e-43a4332d1742 admin admin] The token cache is being invalidate because project 931d44e7d8c84dad9aa3c64eafd899d8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81677) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 10 12:22:36.748082 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-657176b4-cb84-471d-af2e-43a4332d1742 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:22:36.748082 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-657176b4-cb84-471d-af2e-43a4332d1742 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:22:36.748082 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1420/2821] 10.222.0.46 () {58 vars in 1233 bytes} [Mon Oct 10 12:22:36 2022] DELETE /identity/v3/projects/931d44e7d8c84dad9aa3c64eafd899d8 => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:22:36.752949 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39fccfd-8201-4190-9a98-76f409ba35bf None None] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:22:36.753437 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39fccfd-8201-4190-9a98-76f409ba35bf None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:22:36.753800 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39fccfd-8201-4190-9a98-76f409ba35bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:22:36.774582 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.auth.core [None req-b39fccfd-8201-4190-9a98-76f409ba35bf None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:22:36.775849 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-b39fccfd-8201-4190-9a98-76f409ba35bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:22:36.837716 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1402/2822] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:22:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 10 12:22:36.846093 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a62b24-5c09-45ec-b073-7f642ff6ae90 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:22:36.848016 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a62b24-5c09-45ec-b073-7f642ff6ae90 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:22:36.848016 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a62b24-5c09-45ec-b073-7f642ff6ae90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:22:36.854001 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39fccfd-8201-4190-9a98-76f409ba35bf None None] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:22:36.857095 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-b39fccfd-8201-4190-9a98-76f409ba35bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:22:36.867867 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-96a62b24-5c09-45ec-b073-7f642ff6ae90 None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:22:36.868047 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39fccfd-8201-4190-9a98-76f409ba35bf None None] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:22:36.869070 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-96a62b24-5c09-45ec-b073-7f642ff6ae90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:22:36.872775 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d34612b-3026-412b-872e-2da822f60622 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:22:36.882421 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d34612b-3026-412b-872e-2da822f60622 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:22:36.882421 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d34612b-3026-412b-872e-2da822f60622 admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:22:36.882421 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d34612b-3026-412b-872e-2da822f60622 admin admin] PATH_INFO: `/v3/users/2df714a3944a471fbecf2ac37e7e7e45` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:22:36.882421 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d34612b-3026-412b-872e-2da822f60622 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2df714a3944a471fbecf2ac37e7e7e45)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:22:36.882421 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d34612b-3026-412b-872e-2da822f60622 admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:22:36.882930 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1421/2823] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:22:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:22:36.885322 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a62b24-5c09-45ec-b073-7f642ff6ae90 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:22:36.894939 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a62b24-5c09-45ec-b073-7f642ff6ae90 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:22:36.899691 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ba39e1-ec58-451f-8b70-b5e38c3d1dfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:22:36.900288 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ba39e1-ec58-451f-8b70-b5e38c3d1dfd admin admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:22:36.900567 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ba39e1-ec58-451f-8b70-b5e38c3d1dfd admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:22:36.900839 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ba39e1-ec58-451f-8b70-b5e38c3d1dfd admin admin] PATH_INFO: `/v3/domains` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:22:36.901485 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ba39e1-ec58-451f-8b70-b5e38c3d1dfd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 12:22:36.901784 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ba39e1-ec58-451f-8b70-b5e38c3d1dfd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:22:36.903935 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ba39e1-ec58-451f-8b70-b5e38c3d1dfd admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:22:36.915875 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1422/2824] 10.222.0.46 () {58 vars in 1153 bytes} [Mon Oct 10 12:22:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 12:22:36.920744 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ba39e1-ec58-451f-8b70-b5e38c3d1dfd admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:22:36.930539 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ba39e1-ec58-451f-8b70-b5e38c3d1dfd admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:22:36.935178 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e92f32-756f-49e1-8f5b-a00effa630dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:22:36.935752 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e92f32-756f-49e1-8f5b-a00effa630dd admin admin] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:22:36.936032 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e92f32-756f-49e1-8f5b-a00effa630dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:22:36.936464 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e92f32-756f-49e1-8f5b-a00effa630dd admin admin] PATH_INFO: `/v3/projects` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:22:36.936912 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e92f32-756f-49e1-8f5b-a00effa630dd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:22:36.938824 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e92f32-756f-49e1-8f5b-a00effa630dd admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:22:37.050141 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1423/2825] 10.222.0.46 () {60 vars in 1153 bytes} [Mon Oct 10 12:22:36 2022] POST /identity/v3/projects => generated 399 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 10 12:22:37.053167 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e92f32-756f-49e1-8f5b-a00effa630dd admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:22:37.064183 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-5d34612b-3026-412b-872e-2da822f60622 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2df714a3944a471fbecf2ac37e7e7e45'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:22:37.066233 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:22:37.071677 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-5d34612b-3026-412b-872e-2da822f60622 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2df714a3944a471fbecf2ac37e7e7e45'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:22:37.072385 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-5d34612b-3026-412b-872e-2da822f60622 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2df714a3944a471fbecf2ac37e7e7e45'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:22:37.105835 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:22:37.122680 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e92f32-756f-49e1-8f5b-a00effa630dd admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:22:37.127421 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640637ce-666f-4420-9e01-98ca39347ff5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:22:37.128031 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640637ce-666f-4420-9e01-98ca39347ff5 admin admin] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:22:37.128310 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640637ce-666f-4420-9e01-98ca39347ff5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:22:37.128582 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640637ce-666f-4420-9e01-98ca39347ff5 admin admin] PATH_INFO: `/v3/users` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:22:37.129214 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640637ce-666f-4420-9e01-98ca39347ff5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:22:37.131160 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640637ce-666f-4420-9e01-98ca39347ff5 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:22:37.138229 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-5d34612b-3026-412b-872e-2da822f60622 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2df714a3944a471fbecf2ac37e7e7e45'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:22:37.151415 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-5d34612b-3026-412b-872e-2da822f60622 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2df714a3944a471fbecf2ac37e7e7e45'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:22:37.158644 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1403/2826] 10.222.0.46 () {58 vars in 1224 bytes} [Mon Oct 10 12:22:36 2022] DELETE /identity/v3/users/2df714a3944a471fbecf2ac37e7e7e45 => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 10 12:22:37.163135 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0034480f-7ef6-471e-bb12-ef39b2cccfca None None] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:22:37.163418 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0034480f-7ef6-471e-bb12-ef39b2cccfca None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:22:37.163693 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0034480f-7ef6-471e-bb12-ef39b2cccfca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:22:37.183641 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.auth.core [None req-0034480f-7ef6-471e-bb12-ef39b2cccfca None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:22:37.185221 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-0034480f-7ef6-471e-bb12-ef39b2cccfca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:22:37.244248 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1424/2827] 10.222.0.46 () {60 vars in 1144 bytes} [Mon Oct 10 12:22:37 2022] POST /identity/v3/users => generated 300 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 10 12:22:37.245617 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640637ce-666f-4420-9e01-98ca39347ff5 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:22:37.246894 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1404/2828] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:22:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 10 12:22:37.247274 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-640637ce-666f-4420-9e01-98ca39347ff5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:22:37.259970 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640637ce-666f-4420-9e01-98ca39347ff5 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:22:37.265225 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1537fab9-5f3a-4bc2-bcee-5d3566d43033 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:22:37.265842 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1537fab9-5f3a-4bc2-bcee-5d3566d43033 admin admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:22:37.266125 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1537fab9-5f3a-4bc2-bcee-5d3566d43033 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:22:37.266400 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1537fab9-5f3a-4bc2-bcee-5d3566d43033 admin admin] PATH_INFO: `/v3/roles` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:22:37.267090 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1537fab9-5f3a-4bc2-bcee-5d3566d43033 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:22:37.268920 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0034480f-7ef6-471e-bb12-ef39b2cccfca None None] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:22:37.269049 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1537fab9-5f3a-4bc2-bcee-5d3566d43033 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:22:37.270354 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-0034480f-7ef6-471e-bb12-ef39b2cccfca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:22:37.283473 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1425/2829] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 10 12:22:37 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 10 12:22:37.284718 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1537fab9-5f3a-4bc2-bcee-5d3566d43033 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:22:37.294680 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1537fab9-5f3a-4bc2-bcee-5d3566d43033 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:22:37.299412 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69ee61d-49df-4f8f-afbb-675bc39149cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:22:37.300009 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69ee61d-49df-4f8f-afbb-675bc39149cb admin admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:22:37.300360 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69ee61d-49df-4f8f-afbb-675bc39149cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:22:37.300565 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69ee61d-49df-4f8f-afbb-675bc39149cb admin admin] PATH_INFO: `/v3/roles` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:22:37.301197 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69ee61d-49df-4f8f-afbb-675bc39149cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:22:37.303026 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69ee61d-49df-4f8f-afbb-675bc39149cb admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:22:37.317976 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1426/2830] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 10 12:22:37 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 12:22:37.318998 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69ee61d-49df-4f8f-afbb-675bc39149cb admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:22:37.328738 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0034480f-7ef6-471e-bb12-ef39b2cccfca None None] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:22:37.329431 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69ee61d-49df-4f8f-afbb-675bc39149cb admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:22:37.333599 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f747f1ae-7a9b-4153-8947-694461f82eb0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58b1c7c85302421cba823ace4201861f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad3772f7b6a47c28b7ba37a9316a3c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:22:37.334163 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72af6336-252a-484c-9b31-673a474c6821 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:22:37.334285 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f747f1ae-7a9b-4153-8947-694461f82eb0 service neutron] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:22:37.334651 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f747f1ae-7a9b-4153-8947-694461f82eb0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:22:37.336245 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72af6336-252a-484c-9b31-673a474c6821 admin admin] REQUEST_METHOD: `PUT` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:22:37.336352 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f747f1ae-7a9b-4153-8947-694461f82eb0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:22:37.336503 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72af6336-252a-484c-9b31-673a474c6821 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:22:37.336503 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72af6336-252a-484c-9b31-673a474c6821 admin admin] PATH_INFO: `/v3/projects/a8a9da437897418f84ca0620a83ab533/users/70cb8d9a935d483389b9bcee2678e925/roles/b2cb2a6e818e47338ce071e6fedfc673` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:22:37.344956 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f747f1ae-7a9b-4153-8947-694461f82eb0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:22:37.346369 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f747f1ae-7a9b-4153-8947-694461f82eb0 service neutron] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:22:37.351781 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72af6336-252a-484c-9b31-673a474c6821 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a8a9da437897418f84ca0620a83ab533, user_id=70cb8d9a935d483389b9bcee2678e925, role_id=b2cb2a6e818e47338ce071e6fedfc673)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:22:37.354477 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72af6336-252a-484c-9b31-673a474c6821 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:22:37.356065 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1405/2831] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 10 12:22:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 10 12:22:37.427562 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1427/2832] 10.222.0.46 () {60 vars in 1483 bytes} [Mon Oct 10 12:22:37 2022] PUT /identity/v3/projects/a8a9da437897418f84ca0620a83ab533/users/70cb8d9a935d483389b9bcee2678e925/roles/b2cb2a6e818e47338ce071e6fedfc673 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:22:37.438799 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc821e5-b7f8-4363-ba12-76673d7e3a38 None None] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:22:37.439090 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc821e5-b7f8-4363-ba12-76673d7e3a38 None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:22:37.439420 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc821e5-b7f8-4363-ba12-76673d7e3a38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:22:37.476506 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.auth.core [None req-2fc821e5-b7f8-4363-ba12-76673d7e3a38 None None] MFA Rules not processed for user `70cb8d9a935d483389b9bcee2678e925`. Rule list: `[]` (Enabled: `True`). {{(pid=81676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:22:37.477795 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-2fc821e5-b7f8-4363-ba12-76673d7e3a38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:22:37.545793 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:22:37.549979 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1406/2833] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:22:37 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:22:37.564121 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72af6336-252a-484c-9b31-673a474c6821 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:22:37.575064 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-72af6336-252a-484c-9b31-673a474c6821 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:22:37.630519 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72af6336-252a-484c-9b31-673a474c6821 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:22:37.634411 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff31014-23b7-49a3-94ef-b241c51d4ef6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3ae093470484b93b23dcfae1657df9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad3772f7b6a47c28b7ba37a9316a3c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:22:37.635049 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff31014-23b7-49a3-94ef-b241c51d4ef6 service cinder] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:22:37.635331 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff31014-23b7-49a3-94ef-b241c51d4ef6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:22:37.635601 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff31014-23b7-49a3-94ef-b241c51d4ef6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:22:37.645443 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff31014-23b7-49a3-94ef-b241c51d4ef6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:22:37.646906 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff31014-23b7-49a3-94ef-b241c51d4ef6 service cinder] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:22:37.656906 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1428/2834] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 10 12:22:37 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 10 12:22:37.791367 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc821e5-b7f8-4363-ba12-76673d7e3a38 None None] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:22:37.792866 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-2fc821e5-b7f8-4363-ba12-76673d7e3a38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:22:37.864774 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc821e5-b7f8-4363-ba12-76673d7e3a38 None None] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:22:37.869377 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f5b45c-78d2-40ac-8f2d-3f831d3885de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:22:37.870017 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f5b45c-78d2-40ac-8f2d-3f831d3885de admin admin] REQUEST_METHOD: `DELETE` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:22:37.870298 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f5b45c-78d2-40ac-8f2d-3f831d3885de admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:22:37.870571 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f5b45c-78d2-40ac-8f2d-3f831d3885de admin admin] PATH_INFO: `/v3/projects/2406e23f06df493ab9afcba4b0567de1` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:22:37.871762 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f5b45c-78d2-40ac-8f2d-3f831d3885de admin admin] RBAC: Authorizing `identity:delete_project(project_id=2406e23f06df493ab9afcba4b0567de1)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:22:37.874286 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f5b45c-78d2-40ac-8f2d-3f831d3885de admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:22:37.964638 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-23f5b45c-78d2-40ac-8f2d-3f831d3885de admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2406e23f06df493ab9afcba4b0567de1', 'request_id': 'req-23f5b45c-78d2-40ac-8f2d-3f831d3885de'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:22:37.970522 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-23f5b45c-78d2-40ac-8f2d-3f831d3885de admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2406e23f06df493ab9afcba4b0567de1', 'request_id': 'req-23f5b45c-78d2-40ac-8f2d-3f831d3885de'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:22:37.976763 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-23f5b45c-78d2-40ac-8f2d-3f831d3885de admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2406e23f06df493ab9afcba4b0567de1', 'request_id': 'req-23f5b45c-78d2-40ac-8f2d-3f831d3885de'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:22:38.080011 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-23f5b45c-78d2-40ac-8f2d-3f831d3885de admin admin] The token cache is being invalidate because project 2406e23f06df493ab9afcba4b0567de1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81676) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 10 12:22:38.080219 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-23f5b45c-78d2-40ac-8f2d-3f831d3885de admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:22:38.080840 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-23f5b45c-78d2-40ac-8f2d-3f831d3885de admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:22:38.082371 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1407/2835] 10.222.0.46 () {58 vars in 1233 bytes} [Mon Oct 10 12:22:37 2022] DELETE /identity/v3/projects/2406e23f06df493ab9afcba4b0567de1 => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:22:38.086539 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff31014-23b7-49a3-94ef-b241c51d4ef6 service cinder] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:22:38.097532 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff31014-23b7-49a3-94ef-b241c51d4ef6 service cinder] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:22:38.159565 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf32415-8a09-49a5-bff1-2fd03ee24e7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:22:38.160295 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf32415-8a09-49a5-bff1-2fd03ee24e7e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:22:38.160577 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf32415-8a09-49a5-bff1-2fd03ee24e7e admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:22:38.160855 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf32415-8a09-49a5-bff1-2fd03ee24e7e admin admin] PATH_INFO: `/v3/users/5a858c7a10b948e78600099de4a02ed1` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:22:38.162006 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf32415-8a09-49a5-bff1-2fd03ee24e7e admin admin] RBAC: Authorizing `identity:delete_user(user_id=5a858c7a10b948e78600099de4a02ed1)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:22:38.163989 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf32415-8a09-49a5-bff1-2fd03ee24e7e admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:22:38.371658 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=91,keystone:INSERT=4,keystone:DELETE=5 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:22:38.397628 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-3cf32415-8a09-49a5-bff1-2fd03ee24e7e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5a858c7a10b948e78600099de4a02ed1'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:22:38.405497 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-3cf32415-8a09-49a5-bff1-2fd03ee24e7e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5a858c7a10b948e78600099de4a02ed1'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:22:38.406209 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-3cf32415-8a09-49a5-bff1-2fd03ee24e7e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5a858c7a10b948e78600099de4a02ed1'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:22:38.492401 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-3cf32415-8a09-49a5-bff1-2fd03ee24e7e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5a858c7a10b948e78600099de4a02ed1'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:22:38.506809 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-3cf32415-8a09-49a5-bff1-2fd03ee24e7e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5a858c7a10b948e78600099de4a02ed1'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:22:38.512753 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1429/2836] 10.222.0.46 () {58 vars in 1224 bytes} [Mon Oct 10 12:22:38 2022] DELETE /identity/v3/users/5a858c7a10b948e78600099de4a02ed1 => generated 0 bytes in 427 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:22:38.942734 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f5b45c-78d2-40ac-8f2d-3f831d3885de admin admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:22:38.944153 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-23f5b45c-78d2-40ac-8f2d-3f831d3885de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:22:38.959253 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f5b45c-78d2-40ac-8f2d-3f831d3885de admin admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:22:39.019767 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245b9ef3-5716-4a9c-81c8-129afc4149b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:22:39.020421 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245b9ef3-5716-4a9c-81c8-129afc4149b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:22:39.020706 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245b9ef3-5716-4a9c-81c8-129afc4149b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:22:39.020986 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245b9ef3-5716-4a9c-81c8-129afc4149b4 admin admin] PATH_INFO: `/v3/projects/840679fe62464eb8a84ef84970111716` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:22:39.022234 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245b9ef3-5716-4a9c-81c8-129afc4149b4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=840679fe62464eb8a84ef84970111716)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:22:39.024190 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245b9ef3-5716-4a9c-81c8-129afc4149b4 admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:22:39.141998 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:22:39.154316 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-245b9ef3-5716-4a9c-81c8-129afc4149b4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '840679fe62464eb8a84ef84970111716', 'request_id': 'req-245b9ef3-5716-4a9c-81c8-129afc4149b4'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:22:39.160195 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-245b9ef3-5716-4a9c-81c8-129afc4149b4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '840679fe62464eb8a84ef84970111716', 'request_id': 'req-245b9ef3-5716-4a9c-81c8-129afc4149b4'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:22:39.166508 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-245b9ef3-5716-4a9c-81c8-129afc4149b4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '840679fe62464eb8a84ef84970111716', 'request_id': 'req-245b9ef3-5716-4a9c-81c8-129afc4149b4'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:22:39.246526 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-245b9ef3-5716-4a9c-81c8-129afc4149b4 admin admin] The token cache is being invalidate because project 840679fe62464eb8a84ef84970111716 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81676) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 10 12:22:39.246797 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-245b9ef3-5716-4a9c-81c8-129afc4149b4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:22:39.248031 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-245b9ef3-5716-4a9c-81c8-129afc4149b4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:22:39.249814 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1408/2837] 10.222.0.46 () {58 vars in 1233 bytes} [Mon Oct 10 12:22:38 2022] DELETE /identity/v3/projects/840679fe62464eb8a84ef84970111716 => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:22:39.256660 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ea30cc-e580-4269-85e6-357f2570610e None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:22:39.257460 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ea30cc-e580-4269-85e6-357f2570610e None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:22:39.259953 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ea30cc-e580-4269-85e6-357f2570610e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:22:39.280084 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-85ea30cc-e580-4269-85e6-357f2570610e None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:22:39.281334 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-85ea30cc-e580-4269-85e6-357f2570610e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:22:39.346346 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1430/2838] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:22:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:22:39.355840 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d6c2dc-50bc-44a3-ae3c-648ae230de16 None None] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:22:39.356124 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d6c2dc-50bc-44a3-ae3c-648ae230de16 None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:22:39.356428 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d6c2dc-50bc-44a3-ae3c-648ae230de16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:22:39.378255 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.auth.core [None req-e0d6c2dc-50bc-44a3-ae3c-648ae230de16 None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:22:39.380389 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-e0d6c2dc-50bc-44a3-ae3c-648ae230de16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:22:39.388543 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1409/2839] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:22:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:22:39.393115 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ea30cc-e580-4269-85e6-357f2570610e None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:22:39.403748 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ea30cc-e580-4269-85e6-357f2570610e None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:22:39.408506 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e25327-23dd-4cdc-8400-96164dbb9391 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:22:39.409182 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e25327-23dd-4cdc-8400-96164dbb9391 admin admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:22:39.409398 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e25327-23dd-4cdc-8400-96164dbb9391 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:22:39.409672 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e25327-23dd-4cdc-8400-96164dbb9391 admin admin] PATH_INFO: `/v3/domains` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:22:39.410328 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e25327-23dd-4cdc-8400-96164dbb9391 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 12:22:39.410658 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e25327-23dd-4cdc-8400-96164dbb9391 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:22:39.412301 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e25327-23dd-4cdc-8400-96164dbb9391 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:22:39.425946 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1431/2840] 10.222.0.46 () {58 vars in 1153 bytes} [Mon Oct 10 12:22:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 12:22:39.430541 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d6c2dc-50bc-44a3-ae3c-648ae230de16 None None] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:22:39.440725 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d6c2dc-50bc-44a3-ae3c-648ae230de16 None None] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:22:39.445426 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbeb38aa-662b-4d3a-93ec-67d78afa93fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:22:39.446043 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbeb38aa-662b-4d3a-93ec-67d78afa93fc admin admin] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:22:39.446320 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbeb38aa-662b-4d3a-93ec-67d78afa93fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:22:39.446590 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbeb38aa-662b-4d3a-93ec-67d78afa93fc admin admin] PATH_INFO: `/v3/projects` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:22:39.447283 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbeb38aa-662b-4d3a-93ec-67d78afa93fc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:22:39.449379 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbeb38aa-662b-4d3a-93ec-67d78afa93fc admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:22:39.577423 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1410/2841] 10.222.0.46 () {60 vars in 1153 bytes} [Mon Oct 10 12:22:39 2022] POST /identity/v3/projects => generated 367 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 12:22:39.582332 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e25327-23dd-4cdc-8400-96164dbb9391 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:22:39.600502 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e25327-23dd-4cdc-8400-96164dbb9391 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:22:39.661165 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454de5b6-4f06-4891-8866-48ca83424fea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:22:39.661784 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454de5b6-4f06-4891-8866-48ca83424fea admin admin] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:22:39.662064 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454de5b6-4f06-4891-8866-48ca83424fea admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:22:39.662338 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454de5b6-4f06-4891-8866-48ca83424fea admin admin] PATH_INFO: `/v3/users` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:22:39.663021 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454de5b6-4f06-4891-8866-48ca83424fea admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:22:39.664966 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454de5b6-4f06-4891-8866-48ca83424fea admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:22:39.793150 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1432/2842] 10.222.0.46 () {60 vars in 1144 bytes} [Mon Oct 10 12:22:39 2022] POST /identity/v3/users => generated 284 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 12:22:39.798392 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbeb38aa-662b-4d3a-93ec-67d78afa93fc admin admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:22:39.809245 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbeb38aa-662b-4d3a-93ec-67d78afa93fc admin admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:22:39.813857 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131c2c5d-a0d3-48b9-9731-49083c6fe57c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:22:39.819212 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131c2c5d-a0d3-48b9-9731-49083c6fe57c admin admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:22:39.819212 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131c2c5d-a0d3-48b9-9731-49083c6fe57c admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:22:39.819212 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131c2c5d-a0d3-48b9-9731-49083c6fe57c admin admin] PATH_INFO: `/v3/roles` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:22:39.819212 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131c2c5d-a0d3-48b9-9731-49083c6fe57c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:22:39.819212 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131c2c5d-a0d3-48b9-9731-49083c6fe57c admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:22:39.829533 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1411/2843] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 10 12:22:39 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 12:22:39.833924 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454de5b6-4f06-4891-8866-48ca83424fea admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:22:39.844886 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454de5b6-4f06-4891-8866-48ca83424fea admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:22:39.849592 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411139b3-23af-49dd-b079-8b1194397a61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:22:39.850189 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411139b3-23af-49dd-b079-8b1194397a61 admin admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:22:39.850467 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411139b3-23af-49dd-b079-8b1194397a61 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:22:39.850772 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411139b3-23af-49dd-b079-8b1194397a61 admin admin] PATH_INFO: `/v3/roles` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:22:39.851413 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411139b3-23af-49dd-b079-8b1194397a61 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:22:39.852994 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411139b3-23af-49dd-b079-8b1194397a61 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:22:39.864362 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1433/2844] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 10 12:22:39 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 12:22:39.868633 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131c2c5d-a0d3-48b9-9731-49083c6fe57c admin admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:22:39.878580 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131c2c5d-a0d3-48b9-9731-49083c6fe57c admin admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:22:39.883405 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecc43c5-2629-44ad-8273-2efe40fa029a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:22:39.884053 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecc43c5-2629-44ad-8273-2efe40fa029a admin admin] REQUEST_METHOD: `PUT` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:22:39.884337 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecc43c5-2629-44ad-8273-2efe40fa029a admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:22:39.884614 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecc43c5-2629-44ad-8273-2efe40fa029a admin admin] PATH_INFO: `/v3/projects/a404253b0e534b208d90650f943e9ae3/users/524a1520aa6e4444a569977d5aed9e60/roles/b2cb2a6e818e47338ce071e6fedfc673` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:22:39.901139 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fecc43c5-2629-44ad-8273-2efe40fa029a admin admin] RBAC: Authorizing `identity:create_grant(project_id=a404253b0e534b208d90650f943e9ae3, user_id=524a1520aa6e4444a569977d5aed9e60, role_id=b2cb2a6e818e47338ce071e6fedfc673)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:22:39.905309 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fecc43c5-2629-44ad-8273-2efe40fa029a admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:22:40.004972 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1412/2845] 10.222.0.46 () {60 vars in 1483 bytes} [Mon Oct 10 12:22:39 2022] PUT /identity/v3/projects/a404253b0e534b208d90650f943e9ae3/users/524a1520aa6e4444a569977d5aed9e60/roles/b2cb2a6e818e47338ce071e6fedfc673 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:22:40.015886 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957415a9-bbb3-485b-842c-1790b23d8c80 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:22:40.016338 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957415a9-bbb3-485b-842c-1790b23d8c80 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:22:40.016737 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957415a9-bbb3-485b-842c-1790b23d8c80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:22:40.047489 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-957415a9-bbb3-485b-842c-1790b23d8c80 None None] MFA Rules not processed for user `524a1520aa6e4444a569977d5aed9e60`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:22:40.048769 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-957415a9-bbb3-485b-842c-1790b23d8c80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:22:40.052999 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:22:40.120158 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1434/2846] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:22:40 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:22:40.125031 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecc43c5-2629-44ad-8273-2efe40fa029a admin admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:22:40.193474 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecc43c5-2629-44ad-8273-2efe40fa029a admin admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:22:40.198052 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263caade-5604-4cf3-92c1-3aafe613ad30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:22:40.200191 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263caade-5604-4cf3-92c1-3aafe613ad30 admin admin] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:22:40.200191 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263caade-5604-4cf3-92c1-3aafe613ad30 admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:22:40.200191 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263caade-5604-4cf3-92c1-3aafe613ad30 admin admin] PATH_INFO: `/v3/projects` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:22:40.200191 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263caade-5604-4cf3-92c1-3aafe613ad30 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:22:40.202280 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263caade-5604-4cf3-92c1-3aafe613ad30 admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:22:40.312597 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1413/2847] 10.222.0.46 () {60 vars in 1153 bytes} [Mon Oct 10 12:22:40 2022] POST /identity/v3/projects => generated 367 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 12:22:40.317218 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957415a9-bbb3-485b-842c-1790b23d8c80 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:22:40.330086 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957415a9-bbb3-485b-842c-1790b23d8c80 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:22:40.400298 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318fb3ed-9431-4a77-9556-b99140a8f5cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:22:40.400927 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318fb3ed-9431-4a77-9556-b99140a8f5cd admin admin] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:22:40.401209 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318fb3ed-9431-4a77-9556-b99140a8f5cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:22:40.401482 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318fb3ed-9431-4a77-9556-b99140a8f5cd admin admin] PATH_INFO: `/v3/users` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:22:40.402146 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318fb3ed-9431-4a77-9556-b99140a8f5cd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:22:40.405085 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318fb3ed-9431-4a77-9556-b99140a8f5cd admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:22:40.521989 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1435/2848] 10.222.0.46 () {60 vars in 1144 bytes} [Mon Oct 10 12:22:40 2022] POST /identity/v3/users => generated 290 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 12:22:40.522398 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263caade-5604-4cf3-92c1-3aafe613ad30 admin admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:22:40.535129 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263caade-5604-4cf3-92c1-3aafe613ad30 admin admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:22:40.541104 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ce3d5e-29c4-4819-811b-9e38c60d78b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:22:40.541806 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ce3d5e-29c4-4819-811b-9e38c60d78b0 admin admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:22:40.542099 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ce3d5e-29c4-4819-811b-9e38c60d78b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:22:40.542388 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ce3d5e-29c4-4819-811b-9e38c60d78b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:22:40.543084 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ce3d5e-29c4-4819-811b-9e38c60d78b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:22:40.544713 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ce3d5e-29c4-4819-811b-9e38c60d78b0 admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:22:40.559797 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1414/2849] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 10 12:22:40 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 12:22:40.565891 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318fb3ed-9431-4a77-9556-b99140a8f5cd admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:22:40.594017 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318fb3ed-9431-4a77-9556-b99140a8f5cd admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:22:40.599273 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40572887-8fdd-49b0-9775-4c40a968aeb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:22:40.599948 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40572887-8fdd-49b0-9775-4c40a968aeb0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:22:40.600225 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40572887-8fdd-49b0-9775-4c40a968aeb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:22:40.600501 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40572887-8fdd-49b0-9775-4c40a968aeb0 admin admin] PATH_INFO: `/v3/projects/e181520c9f404ab78edd28493e0dd7e7/users/46c7cc97ee8140be92b5134e29a01cfa/roles/0d822a0674b6495fac623877a3b38025` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:22:40.621540 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40572887-8fdd-49b0-9775-4c40a968aeb0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e181520c9f404ab78edd28493e0dd7e7, user_id=46c7cc97ee8140be92b5134e29a01cfa, role_id=0d822a0674b6495fac623877a3b38025)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:22:40.625641 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40572887-8fdd-49b0-9775-4c40a968aeb0 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:22:40.748518 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1436/2850] 10.222.0.46 () {60 vars in 1483 bytes} [Mon Oct 10 12:22:40 2022] PUT /identity/v3/projects/e181520c9f404ab78edd28493e0dd7e7/users/46c7cc97ee8140be92b5134e29a01cfa/roles/0d822a0674b6495fac623877a3b38025 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 10 12:22:40.753238 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ce3d5e-29c4-4819-811b-9e38c60d78b0 admin admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:22:40.764707 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ce3d5e-29c4-4819-811b-9e38c60d78b0 admin admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:22:40.797677 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:22:40.829270 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057854f3-ab65-4d4f-a697-604e593be04e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:22:40.829801 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057854f3-ab65-4d4f-a697-604e593be04e admin admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:22:40.830083 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057854f3-ab65-4d4f-a697-604e593be04e admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:22:40.830351 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057854f3-ab65-4d4f-a697-604e593be04e admin admin] PATH_INFO: `/v3/roles` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:22:40.831150 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057854f3-ab65-4d4f-a697-604e593be04e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:22:40.832792 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057854f3-ab65-4d4f-a697-604e593be04e admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:22:40.843774 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1415/2851] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 10 12:22:40 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 12:22:40.848040 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40572887-8fdd-49b0-9775-4c40a968aeb0 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:22:40.859318 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40572887-8fdd-49b0-9775-4c40a968aeb0 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:22:40.868668 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f884e7e-38cc-425d-8f90-b64947d413de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:22:40.868668 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f884e7e-38cc-425d-8f90-b64947d413de admin admin] REQUEST_METHOD: `PUT` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:22:40.868668 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f884e7e-38cc-425d-8f90-b64947d413de admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:22:40.868668 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f884e7e-38cc-425d-8f90-b64947d413de admin admin] PATH_INFO: `/v3/projects/e181520c9f404ab78edd28493e0dd7e7/users/46c7cc97ee8140be92b5134e29a01cfa/roles/0d822a0674b6495fac623877a3b38025` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:22:40.869445 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f884e7e-38cc-425d-8f90-b64947d413de admin admin] RBAC: Authorizing `identity:create_grant(project_id=e181520c9f404ab78edd28493e0dd7e7, user_id=46c7cc97ee8140be92b5134e29a01cfa, role_id=0d822a0674b6495fac623877a3b38025)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:22:40.870314 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f884e7e-38cc-425d-8f90-b64947d413de admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:22:40.882885 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1437/2852] 10.222.0.46 () {60 vars in 1483 bytes} [Mon Oct 10 12:22:40 2022] PUT /identity/v3/projects/e181520c9f404ab78edd28493e0dd7e7/users/46c7cc97ee8140be92b5134e29a01cfa/roles/0d822a0674b6495fac623877a3b38025 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:22:40.893301 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab192a4d-a247-40b3-a2fb-20e6f9d017c9 None None] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:22:40.893586 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab192a4d-a247-40b3-a2fb-20e6f9d017c9 None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:22:40.893864 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab192a4d-a247-40b3-a2fb-20e6f9d017c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:22:40.922799 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.auth.core [None req-ab192a4d-a247-40b3-a2fb-20e6f9d017c9 None None] MFA Rules not processed for user `46c7cc97ee8140be92b5134e29a01cfa`. Rule list: `[]` (Enabled: `True`). {{(pid=81676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:22:40.927188 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-ab192a4d-a247-40b3-a2fb-20e6f9d017c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:22:41.003999 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1416/2853] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:22:40 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 10 12:22:41.018854 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f884e7e-38cc-425d-8f90-b64947d413de admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:22:41.020732 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-0f884e7e-38cc-425d-8f90-b64947d413de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:22:41.085887 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f884e7e-38cc-425d-8f90-b64947d413de admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:22:41.091562 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6c1baf-9f00-4526-924c-c38a88bf69d9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3ae093470484b93b23dcfae1657df9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad3772f7b6a47c28b7ba37a9316a3c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:22:41.092194 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6c1baf-9f00-4526-924c-c38a88bf69d9 service cinder] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:22:41.092471 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6c1baf-9f00-4526-924c-c38a88bf69d9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:22:41.092903 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6c1baf-9f00-4526-924c-c38a88bf69d9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:22:41.114968 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6c1baf-9f00-4526-924c-c38a88bf69d9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:22:41.117689 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6c1baf-9f00-4526-924c-c38a88bf69d9 service cinder] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:22:41.128218 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:22:41.137663 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1438/2854] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 10 12:22:41 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:22:41.380442 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab192a4d-a247-40b3-a2fb-20e6f9d017c9 None None] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:22:41.395800 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab192a4d-a247-40b3-a2fb-20e6f9d017c9 None None] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:22:41.399516 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a191a8-f9ce-4bd6-a756-55599a58a99d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3ae093470484b93b23dcfae1657df9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad3772f7b6a47c28b7ba37a9316a3c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:22:41.400116 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a191a8-f9ce-4bd6-a756-55599a58a99d service cinder] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:22:41.400391 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a191a8-f9ce-4bd6-a756-55599a58a99d service cinder] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:22:41.400681 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a191a8-f9ce-4bd6-a756-55599a58a99d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:22:41.459553 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a191a8-f9ce-4bd6-a756-55599a58a99d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:22:41.461236 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a191a8-f9ce-4bd6-a756-55599a58a99d service cinder] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:22:41.472193 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1417/2855] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 10 12:22:41 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:22:48.892482 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6c1baf-9f00-4526-924c-c38a88bf69d9 service cinder] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:22:48.893980 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-5a6c1baf-9f00-4526-924c-c38a88bf69d9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:22:48.963316 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6c1baf-9f00-4526-924c-c38a88bf69d9 service cinder] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:22:48.967938 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8263309-e85c-44b1-bbe3-ea7c92ab8304 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:22:48.969110 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8263309-e85c-44b1-bbe3-ea7c92ab8304 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:22:48.969390 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8263309-e85c-44b1-bbe3-ea7c92ab8304 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:22:48.969666 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8263309-e85c-44b1-bbe3-ea7c92ab8304 admin admin] PATH_INFO: `/v3/users/70cb8d9a935d483389b9bcee2678e925` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:22:48.970883 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8263309-e85c-44b1-bbe3-ea7c92ab8304 admin admin] RBAC: Authorizing `identity:delete_user(user_id=70cb8d9a935d483389b9bcee2678e925)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:22:48.972915 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8263309-e85c-44b1-bbe3-ea7c92ab8304 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:22:49.239495 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-b8263309-e85c-44b1-bbe3-ea7c92ab8304 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '70cb8d9a935d483389b9bcee2678e925'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:22:49.247219 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-b8263309-e85c-44b1-bbe3-ea7c92ab8304 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '70cb8d9a935d483389b9bcee2678e925'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:22:49.247944 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-b8263309-e85c-44b1-bbe3-ea7c92ab8304 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '70cb8d9a935d483389b9bcee2678e925'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:22:49.360876 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-b8263309-e85c-44b1-bbe3-ea7c92ab8304 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '70cb8d9a935d483389b9bcee2678e925'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:22:49.374675 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-b8263309-e85c-44b1-bbe3-ea7c92ab8304 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '70cb8d9a935d483389b9bcee2678e925'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:22:49.380871 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1439/2856] 10.222.0.46 () {58 vars in 1224 bytes} [Mon Oct 10 12:22:48 2022] DELETE /identity/v3/users/70cb8d9a935d483389b9bcee2678e925 => generated 0 bytes in 489 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:22:49.385748 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267639b0-9137-4c6a-90bc-219ac795bb2d None None] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:22:49.386242 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267639b0-9137-4c6a-90bc-219ac795bb2d None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:22:49.386348 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267639b0-9137-4c6a-90bc-219ac795bb2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:22:49.407153 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.auth.core [None req-267639b0-9137-4c6a-90bc-219ac795bb2d None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:22:49.408262 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-267639b0-9137-4c6a-90bc-219ac795bb2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:22:49.458504 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:22:49.478830 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1418/2857] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:22:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 10 12:22:49.497729 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8263309-e85c-44b1-bbe3-ea7c92ab8304 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:22:49.497729 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-b8263309-e85c-44b1-bbe3-ea7c92ab8304 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:22:49.571777 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8263309-e85c-44b1-bbe3-ea7c92ab8304 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:22:49.575374 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e77706-b8be-4ff1-ab18-e05614949ed1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58b1c7c85302421cba823ace4201861f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad3772f7b6a47c28b7ba37a9316a3c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:22:49.576211 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e77706-b8be-4ff1-ab18-e05614949ed1 service neutron] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:22:49.576347 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e77706-b8be-4ff1-ab18-e05614949ed1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:22:49.576514 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e77706-b8be-4ff1-ab18-e05614949ed1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:22:49.592475 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e77706-b8be-4ff1-ab18-e05614949ed1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:22:49.592475 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e77706-b8be-4ff1-ab18-e05614949ed1 service neutron] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:22:49.599929 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1440/2858] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 10 12:22:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:22:50.107450 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267639b0-9137-4c6a-90bc-219ac795bb2d None None] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:22:50.109515 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-267639b0-9137-4c6a-90bc-219ac795bb2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:22:50.126188 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267639b0-9137-4c6a-90bc-219ac795bb2d None None] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:22:50.130194 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99195fa3-2f4a-420e-9394-83a00a81df86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:22:50.130492 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99195fa3-2f4a-420e-9394-83a00a81df86 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:22:50.130705 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99195fa3-2f4a-420e-9394-83a00a81df86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:22:50.130904 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99195fa3-2f4a-420e-9394-83a00a81df86 admin admin] PATH_INFO: `/v3/projects/a8a9da437897418f84ca0620a83ab533` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:22:50.132058 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99195fa3-2f4a-420e-9394-83a00a81df86 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a8a9da437897418f84ca0620a83ab533)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:22:50.134004 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99195fa3-2f4a-420e-9394-83a00a81df86 admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:22:50.291528 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-99195fa3-2f4a-420e-9394-83a00a81df86 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a8a9da437897418f84ca0620a83ab533', 'request_id': 'req-99195fa3-2f4a-420e-9394-83a00a81df86'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:22:50.300724 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-99195fa3-2f4a-420e-9394-83a00a81df86 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a8a9da437897418f84ca0620a83ab533', 'request_id': 'req-99195fa3-2f4a-420e-9394-83a00a81df86'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:22:50.308070 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-99195fa3-2f4a-420e-9394-83a00a81df86 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a8a9da437897418f84ca0620a83ab533', 'request_id': 'req-99195fa3-2f4a-420e-9394-83a00a81df86'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:22:50.377865 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-99195fa3-2f4a-420e-9394-83a00a81df86 admin admin] The token cache is being invalidate because project a8a9da437897418f84ca0620a83ab533 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81676) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 10 12:22:50.378153 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-99195fa3-2f4a-420e-9394-83a00a81df86 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:22:50.379222 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-99195fa3-2f4a-420e-9394-83a00a81df86 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:22:50.380985 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1419/2859] 10.222.0.46 () {58 vars in 1233 bytes} [Mon Oct 10 12:22:50 2022] DELETE /identity/v3/projects/a8a9da437897418f84ca0620a83ab533 => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:22:55.132565 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e77706-b8be-4ff1-ab18-e05614949ed1 service neutron] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:22:55.134275 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-a6e77706-b8be-4ff1-ab18-e05614949ed1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:22:55.149649 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e77706-b8be-4ff1-ab18-e05614949ed1 service neutron] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:22:55.169702 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:22:55.208210 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b647b95a-8aa5-49c1-8aef-b8d7df76f575 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:22:55.208841 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b647b95a-8aa5-49c1-8aef-b8d7df76f575 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:22:55.209118 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b647b95a-8aa5-49c1-8aef-b8d7df76f575 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:22:55.209417 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b647b95a-8aa5-49c1-8aef-b8d7df76f575 admin admin] PATH_INFO: `/v3/users/524a1520aa6e4444a569977d5aed9e60` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:22:55.210853 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b647b95a-8aa5-49c1-8aef-b8d7df76f575 admin admin] RBAC: Authorizing `identity:delete_user(user_id=524a1520aa6e4444a569977d5aed9e60)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:22:55.212750 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b647b95a-8aa5-49c1-8aef-b8d7df76f575 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:22:55.447235 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-b647b95a-8aa5-49c1-8aef-b8d7df76f575 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '524a1520aa6e4444a569977d5aed9e60'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:22:55.455376 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-b647b95a-8aa5-49c1-8aef-b8d7df76f575 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '524a1520aa6e4444a569977d5aed9e60'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:22:55.456097 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-b647b95a-8aa5-49c1-8aef-b8d7df76f575 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '524a1520aa6e4444a569977d5aed9e60'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:22:55.531771 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-b647b95a-8aa5-49c1-8aef-b8d7df76f575 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '524a1520aa6e4444a569977d5aed9e60'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:22:55.544748 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-b647b95a-8aa5-49c1-8aef-b8d7df76f575 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '524a1520aa6e4444a569977d5aed9e60'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:22:55.550652 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1441/2860] 10.222.0.46 () {58 vars in 1224 bytes} [Mon Oct 10 12:22:55 2022] DELETE /identity/v3/users/524a1520aa6e4444a569977d5aed9e60 => generated 0 bytes in 419 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:22:55.555393 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbbd845-44c2-4a73-aaac-891e97a3ff9f None None] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:22:55.555917 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbbd845-44c2-4a73-aaac-891e97a3ff9f None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:22:55.555917 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbbd845-44c2-4a73-aaac-891e97a3ff9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:22:55.575725 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.auth.core [None req-7cbbd845-44c2-4a73-aaac-891e97a3ff9f None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:22:55.576934 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-7cbbd845-44c2-4a73-aaac-891e97a3ff9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:22:55.635840 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1420/2861] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:22:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:22:55.645949 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b647b95a-8aa5-49c1-8aef-b8d7df76f575 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:22:55.647575 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-b647b95a-8aa5-49c1-8aef-b8d7df76f575 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:22:55.705730 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b647b95a-8aa5-49c1-8aef-b8d7df76f575 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:22:55.720593 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1ed44d-d8d0-47b9-ac8f-ad3d54717f5b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58b1c7c85302421cba823ace4201861f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad3772f7b6a47c28b7ba37a9316a3c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:22:55.720593 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1ed44d-d8d0-47b9-ac8f-ad3d54717f5b service neutron] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:22:55.720593 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1ed44d-d8d0-47b9-ac8f-ad3d54717f5b service neutron] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:22:55.720593 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1ed44d-d8d0-47b9-ac8f-ad3d54717f5b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:22:55.720978 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1ed44d-d8d0-47b9-ac8f-ad3d54717f5b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:22:55.721259 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1ed44d-d8d0-47b9-ac8f-ad3d54717f5b service neutron] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:22:55.730603 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1442/2862] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 10 12:22:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:22:56.139251 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbbd845-44c2-4a73-aaac-891e97a3ff9f None None] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:22:56.140696 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-7cbbd845-44c2-4a73-aaac-891e97a3ff9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:22:56.153007 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbbd845-44c2-4a73-aaac-891e97a3ff9f None None] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:22:56.158426 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d394f76-4ad8-4214-9edb-9f1caa3c575c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:22:56.159105 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d394f76-4ad8-4214-9edb-9f1caa3c575c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:22:56.159531 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d394f76-4ad8-4214-9edb-9f1caa3c575c admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:22:56.159815 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d394f76-4ad8-4214-9edb-9f1caa3c575c admin admin] PATH_INFO: `/v3/projects/a404253b0e534b208d90650f943e9ae3` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:22:56.161076 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d394f76-4ad8-4214-9edb-9f1caa3c575c admin admin] RBAC: Authorizing `identity:delete_project(project_id=a404253b0e534b208d90650f943e9ae3)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:22:56.163040 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d394f76-4ad8-4214-9edb-9f1caa3c575c admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:22:56.318350 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-7d394f76-4ad8-4214-9edb-9f1caa3c575c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a404253b0e534b208d90650f943e9ae3', 'request_id': 'req-7d394f76-4ad8-4214-9edb-9f1caa3c575c'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:22:56.324504 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-7d394f76-4ad8-4214-9edb-9f1caa3c575c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a404253b0e534b208d90650f943e9ae3', 'request_id': 'req-7d394f76-4ad8-4214-9edb-9f1caa3c575c'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:22:56.330758 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-7d394f76-4ad8-4214-9edb-9f1caa3c575c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a404253b0e534b208d90650f943e9ae3', 'request_id': 'req-7d394f76-4ad8-4214-9edb-9f1caa3c575c'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:22:56.493220 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:22:56.500480 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-7d394f76-4ad8-4214-9edb-9f1caa3c575c admin admin] The token cache is being invalidate because project a404253b0e534b208d90650f943e9ae3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81676) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 10 12:22:56.500480 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-7d394f76-4ad8-4214-9edb-9f1caa3c575c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:22:56.500480 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-7d394f76-4ad8-4214-9edb-9f1caa3c575c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:22:56.501153 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1421/2863] 10.222.0.46 () {58 vars in 1233 bytes} [Mon Oct 10 12:22:56 2022] DELETE /identity/v3/projects/a404253b0e534b208d90650f943e9ae3 => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:22:56.505626 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1ed44d-d8d0-47b9-ac8f-ad3d54717f5b service neutron] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:22:56.516861 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1ed44d-d8d0-47b9-ac8f-ad3d54717f5b service neutron] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:22:56.560809 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:22:56.576785 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4703620-51a1-4a62-8193-ff32ffb8dd06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:22:56.577427 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4703620-51a1-4a62-8193-ff32ffb8dd06 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:22:56.577737 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4703620-51a1-4a62-8193-ff32ffb8dd06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:22:56.578016 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4703620-51a1-4a62-8193-ff32ffb8dd06 admin admin] PATH_INFO: `/v3/users/46c7cc97ee8140be92b5134e29a01cfa` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:22:56.579700 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4703620-51a1-4a62-8193-ff32ffb8dd06 admin admin] RBAC: Authorizing `identity:delete_user(user_id=46c7cc97ee8140be92b5134e29a01cfa)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:22:56.581662 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4703620-51a1-4a62-8193-ff32ffb8dd06 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:22:56.934736 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-a4703620-51a1-4a62-8193-ff32ffb8dd06 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '46c7cc97ee8140be92b5134e29a01cfa'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:22:56.943011 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-a4703620-51a1-4a62-8193-ff32ffb8dd06 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '46c7cc97ee8140be92b5134e29a01cfa'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:22:56.943950 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-a4703620-51a1-4a62-8193-ff32ffb8dd06 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '46c7cc97ee8140be92b5134e29a01cfa'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:22:57.043590 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-a4703620-51a1-4a62-8193-ff32ffb8dd06 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '46c7cc97ee8140be92b5134e29a01cfa'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:22:57.058294 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-a4703620-51a1-4a62-8193-ff32ffb8dd06 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '46c7cc97ee8140be92b5134e29a01cfa'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:22:57.065037 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1443/2864] 10.222.0.46 () {58 vars in 1224 bytes} [Mon Oct 10 12:22:56 2022] DELETE /identity/v3/users/46c7cc97ee8140be92b5134e29a01cfa => generated 0 bytes in 560 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:22:57.525593 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d394f76-4ad8-4214-9edb-9f1caa3c575c admin admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:22:57.527139 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-7d394f76-4ad8-4214-9edb-9f1caa3c575c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:22:57.544293 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d394f76-4ad8-4214-9edb-9f1caa3c575c admin admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:22:57.606219 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6295996-364a-4907-ade2-1e52d73ece8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:22:57.606893 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6295996-364a-4907-ade2-1e52d73ece8d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:22:57.607177 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6295996-364a-4907-ade2-1e52d73ece8d admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:22:57.607445 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6295996-364a-4907-ade2-1e52d73ece8d admin admin] PATH_INFO: `/v3/projects/e181520c9f404ab78edd28493e0dd7e7` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:22:57.608605 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6295996-364a-4907-ade2-1e52d73ece8d admin admin] RBAC: Authorizing `identity:delete_project(project_id=e181520c9f404ab78edd28493e0dd7e7)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:22:57.610547 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6295996-364a-4907-ade2-1e52d73ece8d admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:22:57.767933 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-d6295996-364a-4907-ade2-1e52d73ece8d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e181520c9f404ab78edd28493e0dd7e7', 'request_id': 'req-d6295996-364a-4907-ade2-1e52d73ece8d'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:22:57.773331 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-d6295996-364a-4907-ade2-1e52d73ece8d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e181520c9f404ab78edd28493e0dd7e7', 'request_id': 'req-d6295996-364a-4907-ade2-1e52d73ece8d'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:22:57.779535 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-d6295996-364a-4907-ade2-1e52d73ece8d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e181520c9f404ab78edd28493e0dd7e7', 'request_id': 'req-d6295996-364a-4907-ade2-1e52d73ece8d'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:22:57.891936 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-d6295996-364a-4907-ade2-1e52d73ece8d admin admin] The token cache is being invalidate because project e181520c9f404ab78edd28493e0dd7e7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81676) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 10 12:22:57.892229 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-d6295996-364a-4907-ade2-1e52d73ece8d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:22:57.892931 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-d6295996-364a-4907-ade2-1e52d73ece8d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:22:57.894515 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1422/2865] 10.222.0.46 () {58 vars in 1233 bytes} [Mon Oct 10 12:22:57 2022] DELETE /identity/v3/projects/e181520c9f404ab78edd28493e0dd7e7 => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:22:57.899445 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86501d90-57ae-4846-b8e4-616671a4c682 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:22:57.900002 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86501d90-57ae-4846-b8e4-616671a4c682 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:22:57.900002 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86501d90-57ae-4846-b8e4-616671a4c682 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:22:57.920584 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-86501d90-57ae-4846-b8e4-616671a4c682 None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:22:57.921789 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-86501d90-57ae-4846-b8e4-616671a4c682 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:22:57.988030 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1444/2866] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:22:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:22:58.056836 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88468a20-9cca-48fe-bca5-d0278fed0e1d None None] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:22:58.057111 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88468a20-9cca-48fe-bca5-d0278fed0e1d None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:22:58.057532 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88468a20-9cca-48fe-bca5-d0278fed0e1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:22:58.087486 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.auth.core [None req-88468a20-9cca-48fe-bca5-d0278fed0e1d None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:22:58.092426 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-88468a20-9cca-48fe-bca5-d0278fed0e1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:22:58.106337 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86501d90-57ae-4846-b8e4-616671a4c682 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:22:58.107355 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1423/2867] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:22:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:22:58.118780 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86501d90-57ae-4846-b8e4-616671a4c682 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:22:58.123620 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8aca0fd-15dd-4d2e-a5ab-c4223ba337f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:22:58.124235 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8aca0fd-15dd-4d2e-a5ab-c4223ba337f1 admin admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:22:58.124515 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8aca0fd-15dd-4d2e-a5ab-c4223ba337f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:22:58.124789 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8aca0fd-15dd-4d2e-a5ab-c4223ba337f1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:22:58.125580 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8aca0fd-15dd-4d2e-a5ab-c4223ba337f1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 12:22:58.125876 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8aca0fd-15dd-4d2e-a5ab-c4223ba337f1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:22:58.128015 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8aca0fd-15dd-4d2e-a5ab-c4223ba337f1 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:22:58.142740 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1445/2868] 10.222.0.46 () {58 vars in 1153 bytes} [Mon Oct 10 12:22:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 12:22:58.145578 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88468a20-9cca-48fe-bca5-d0278fed0e1d None None] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:22:58.155223 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88468a20-9cca-48fe-bca5-d0278fed0e1d None None] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:22:58.160051 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea81ca39-49d8-46df-b179-029bb89fe81d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:22:58.160650 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea81ca39-49d8-46df-b179-029bb89fe81d admin admin] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:22:58.160935 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea81ca39-49d8-46df-b179-029bb89fe81d admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:22:58.161233 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea81ca39-49d8-46df-b179-029bb89fe81d admin admin] PATH_INFO: `/v3/projects` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:22:58.161862 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea81ca39-49d8-46df-b179-029bb89fe81d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:22:58.163795 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea81ca39-49d8-46df-b179-029bb89fe81d admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:22:58.293857 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1424/2869] 10.222.0.46 () {60 vars in 1153 bytes} [Mon Oct 10 12:22:58 2022] POST /identity/v3/projects => generated 385 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 12:22:58.298805 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8aca0fd-15dd-4d2e-a5ab-c4223ba337f1 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:22:58.308742 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8aca0fd-15dd-4d2e-a5ab-c4223ba337f1 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:22:58.358877 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:22:58.370061 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e1cfb9-64e1-4f6e-a022-86ec9b448594 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:22:58.370715 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e1cfb9-64e1-4f6e-a022-86ec9b448594 admin admin] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:22:58.370998 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e1cfb9-64e1-4f6e-a022-86ec9b448594 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:22:58.371272 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e1cfb9-64e1-4f6e-a022-86ec9b448594 admin admin] PATH_INFO: `/v3/users` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:22:58.371942 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e1cfb9-64e1-4f6e-a022-86ec9b448594 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:22:58.373896 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e1cfb9-64e1-4f6e-a022-86ec9b448594 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:22:58.492723 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1446/2870] 10.222.0.46 () {60 vars in 1144 bytes} [Mon Oct 10 12:22:58 2022] POST /identity/v3/users => generated 293 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 12:22:58.497031 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea81ca39-49d8-46df-b179-029bb89fe81d admin admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:22:58.507776 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea81ca39-49d8-46df-b179-029bb89fe81d admin admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:22:58.512569 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18220a11-b2e9-4635-8d74-76d7e085cf1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:22:58.513303 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18220a11-b2e9-4635-8d74-76d7e085cf1f admin admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:22:58.513585 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18220a11-b2e9-4635-8d74-76d7e085cf1f admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:22:58.513862 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18220a11-b2e9-4635-8d74-76d7e085cf1f admin admin] PATH_INFO: `/v3/roles` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:22:58.514651 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18220a11-b2e9-4635-8d74-76d7e085cf1f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:22:58.516412 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18220a11-b2e9-4635-8d74-76d7e085cf1f admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:22:58.526785 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1425/2871] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 10 12:22:58 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 12:22:58.530914 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e1cfb9-64e1-4f6e-a022-86ec9b448594 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:22:58.541437 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e1cfb9-64e1-4f6e-a022-86ec9b448594 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:22:58.546225 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a250168-9a72-4cd1-8c4f-54cddb06854f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:22:58.547429 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a250168-9a72-4cd1-8c4f-54cddb06854f admin admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:22:58.547747 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a250168-9a72-4cd1-8c4f-54cddb06854f admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:22:58.548029 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a250168-9a72-4cd1-8c4f-54cddb06854f admin admin] PATH_INFO: `/v3/roles` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:22:58.548670 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a250168-9a72-4cd1-8c4f-54cddb06854f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:22:58.550264 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a250168-9a72-4cd1-8c4f-54cddb06854f admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:22:58.561104 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1447/2872] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 10 12:22:58 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 10 12:22:58.565330 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18220a11-b2e9-4635-8d74-76d7e085cf1f admin admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:22:58.575672 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18220a11-b2e9-4635-8d74-76d7e085cf1f admin admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:22:58.580602 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963ea272-6b59-4ef4-9f26-5d416f78bd39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:22:58.582799 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963ea272-6b59-4ef4-9f26-5d416f78bd39 admin admin] REQUEST_METHOD: `PUT` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:22:58.582799 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963ea272-6b59-4ef4-9f26-5d416f78bd39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:22:58.582799 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963ea272-6b59-4ef4-9f26-5d416f78bd39 admin admin] PATH_INFO: `/v3/projects/c16116bce0b943bdba7d985219bb18e6/users/af1c597a0cd8489b88109719c14473e7/roles/b2cb2a6e818e47338ce071e6fedfc673` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:22:58.597914 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963ea272-6b59-4ef4-9f26-5d416f78bd39 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c16116bce0b943bdba7d985219bb18e6, user_id=af1c597a0cd8489b88109719c14473e7, role_id=b2cb2a6e818e47338ce071e6fedfc673)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:22:58.600648 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963ea272-6b59-4ef4-9f26-5d416f78bd39 admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:22:58.689563 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1426/2873] 10.222.0.46 () {60 vars in 1483 bytes} [Mon Oct 10 12:22:58 2022] PUT /identity/v3/projects/c16116bce0b943bdba7d985219bb18e6/users/af1c597a0cd8489b88109719c14473e7/roles/b2cb2a6e818e47338ce071e6fedfc673 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:22:58.699659 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cd3d14-d62f-4193-b70a-5823d8aaaa60 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:22:58.699949 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cd3d14-d62f-4193-b70a-5823d8aaaa60 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:22:58.700227 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cd3d14-d62f-4193-b70a-5823d8aaaa60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:22:58.728501 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-57cd3d14-d62f-4193-b70a-5823d8aaaa60 None None] MFA Rules not processed for user `af1c597a0cd8489b88109719c14473e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:22:58.729702 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-57cd3d14-d62f-4193-b70a-5823d8aaaa60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:22:58.798918 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1448/2874] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:22:58 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:22:58.810265 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963ea272-6b59-4ef4-9f26-5d416f78bd39 admin admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:22:58.814070 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-963ea272-6b59-4ef4-9f26-5d416f78bd39 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:22:58.842344 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:22:58.872679 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963ea272-6b59-4ef4-9f26-5d416f78bd39 admin admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:22:58.876613 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44972f2e-6363-4594-aabf-ea4f0ec9b2b5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3ae093470484b93b23dcfae1657df9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad3772f7b6a47c28b7ba37a9316a3c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:22:58.877216 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44972f2e-6363-4594-aabf-ea4f0ec9b2b5 service cinder] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:22:58.877497 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44972f2e-6363-4594-aabf-ea4f0ec9b2b5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:22:58.877768 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44972f2e-6363-4594-aabf-ea4f0ec9b2b5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:22:58.886335 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44972f2e-6363-4594-aabf-ea4f0ec9b2b5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:22:58.888432 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44972f2e-6363-4594-aabf-ea4f0ec9b2b5 service cinder] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:22:58.897530 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1427/2875] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 10 12:22:58 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:22:58.918162 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57cd3d14-d62f-4193-b70a-5823d8aaaa60 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:22:58.983662 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57cd3d14-d62f-4193-b70a-5823d8aaaa60 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:22:58.988758 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0616c5d-1880-4359-9524-4484d1be43c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:22:58.989389 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0616c5d-1880-4359-9524-4484d1be43c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:22:58.989699 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0616c5d-1880-4359-9524-4484d1be43c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:22:58.989970 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0616c5d-1880-4359-9524-4484d1be43c9 admin admin] PATH_INFO: `/v3/users/af1c597a0cd8489b88109719c14473e7` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:22:58.991117 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0616c5d-1880-4359-9524-4484d1be43c9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=af1c597a0cd8489b88109719c14473e7)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:22:58.993407 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0616c5d-1880-4359-9524-4484d1be43c9 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:22:59.158560 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-a0616c5d-1880-4359-9524-4484d1be43c9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'af1c597a0cd8489b88109719c14473e7'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:22:59.166235 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-a0616c5d-1880-4359-9524-4484d1be43c9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'af1c597a0cd8489b88109719c14473e7'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:22:59.167017 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-a0616c5d-1880-4359-9524-4484d1be43c9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'af1c597a0cd8489b88109719c14473e7'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:22:59.175074 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=88,keystone:INSERT=4,keystone:DELETE=8 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:22:59.283993 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-a0616c5d-1880-4359-9524-4484d1be43c9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'af1c597a0cd8489b88109719c14473e7'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:22:59.296976 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-a0616c5d-1880-4359-9524-4484d1be43c9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'af1c597a0cd8489b88109719c14473e7'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:22:59.304378 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1449/2876] 10.222.0.46 () {58 vars in 1224 bytes} [Mon Oct 10 12:22:58 2022] DELETE /identity/v3/users/af1c597a0cd8489b88109719c14473e7 => generated 0 bytes in 387 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:22:59.309011 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eea6c64-a842-49db-94af-7e9c2a12f3fc None None] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:22:59.309307 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eea6c64-a842-49db-94af-7e9c2a12f3fc None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:22:59.309599 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eea6c64-a842-49db-94af-7e9c2a12f3fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:22:59.331826 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.auth.core [None req-7eea6c64-a842-49db-94af-7e9c2a12f3fc None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:22:59.333431 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-7eea6c64-a842-49db-94af-7e9c2a12f3fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:22:59.396413 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1428/2877] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:22:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 10 12:22:59.406856 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0616c5d-1880-4359-9524-4484d1be43c9 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:22:59.408371 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-a0616c5d-1880-4359-9524-4484d1be43c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:22:59.488924 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0616c5d-1880-4359-9524-4484d1be43c9 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:22:59.495880 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbeb9df-75c1-4463-96db-b5c329f7de22 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58b1c7c85302421cba823ace4201861f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad3772f7b6a47c28b7ba37a9316a3c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:22:59.496609 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbeb9df-75c1-4463-96db-b5c329f7de22 service neutron] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:22:59.496985 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbeb9df-75c1-4463-96db-b5c329f7de22 service neutron] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:22:59.497358 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbeb9df-75c1-4463-96db-b5c329f7de22 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:22:59.508514 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffbeb9df-75c1-4463-96db-b5c329f7de22 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:22:59.510055 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffbeb9df-75c1-4463-96db-b5c329f7de22 service neutron] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:22:59.522374 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1450/2878] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 10 12:22:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:22:59.590656 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eea6c64-a842-49db-94af-7e9c2a12f3fc None None] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:22:59.592856 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-7eea6c64-a842-49db-94af-7e9c2a12f3fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:22:59.606223 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eea6c64-a842-49db-94af-7e9c2a12f3fc None None] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:22:59.613384 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d1118b-fb4b-4427-ac03-5b62316df3d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:22:59.614967 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d1118b-fb4b-4427-ac03-5b62316df3d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:22:59.615833 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d1118b-fb4b-4427-ac03-5b62316df3d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:22:59.616178 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d1118b-fb4b-4427-ac03-5b62316df3d1 admin admin] PATH_INFO: `/v3/users/d77069477eb54f2aa70e71de2ada760c` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:22:59.618424 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d1118b-fb4b-4427-ac03-5b62316df3d1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d77069477eb54f2aa70e71de2ada760c)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:22:59.620699 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d1118b-fb4b-4427-ac03-5b62316df3d1 admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:22:59.810880 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-95d1118b-fb4b-4427-ac03-5b62316df3d1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd77069477eb54f2aa70e71de2ada760c'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:22:59.819062 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-95d1118b-fb4b-4427-ac03-5b62316df3d1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd77069477eb54f2aa70e71de2ada760c'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:22:59.819799 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-95d1118b-fb4b-4427-ac03-5b62316df3d1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd77069477eb54f2aa70e71de2ada760c'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:22:59.889567 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-95d1118b-fb4b-4427-ac03-5b62316df3d1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd77069477eb54f2aa70e71de2ada760c'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:22:59.903921 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-95d1118b-fb4b-4427-ac03-5b62316df3d1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd77069477eb54f2aa70e71de2ada760c'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:22:59.911248 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1429/2879] 10.222.0.46 () {58 vars in 1224 bytes} [Mon Oct 10 12:22:59 2022] DELETE /identity/v3/users/d77069477eb54f2aa70e71de2ada760c => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:22:59.915822 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc777926-fd1b-47d3-b3e8-5de541efecbc None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:22:59.916114 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc777926-fd1b-47d3-b3e8-5de541efecbc None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:22:59.916658 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc777926-fd1b-47d3-b3e8-5de541efecbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:22:59.936780 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-fc777926-fd1b-47d3-b3e8-5de541efecbc None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:22:59.937994 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-fc777926-fd1b-47d3-b3e8-5de541efecbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:23:00.005484 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1451/2880] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:22:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:23:00.015901 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d1118b-fb4b-4427-ac03-5b62316df3d1 admin admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:23:00.020903 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-95d1118b-fb4b-4427-ac03-5b62316df3d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:23:00.072009 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:23:00.086563 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d1118b-fb4b-4427-ac03-5b62316df3d1 admin admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:23:00.091038 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f331c4a6-5f08-41a4-bf41-3e871a40e211 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58b1c7c85302421cba823ace4201861f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad3772f7b6a47c28b7ba37a9316a3c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:23:00.091656 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f331c4a6-5f08-41a4-bf41-3e871a40e211 service neutron] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:23:00.091948 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f331c4a6-5f08-41a4-bf41-3e871a40e211 service neutron] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:23:00.092238 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f331c4a6-5f08-41a4-bf41-3e871a40e211 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:23:00.102807 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f331c4a6-5f08-41a4-bf41-3e871a40e211 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:23:00.104236 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f331c4a6-5f08-41a4-bf41-3e871a40e211 service neutron] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:23:00.115225 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1430/2881] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 10 12:23:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 10 12:23:00.137854 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc777926-fd1b-47d3-b3e8-5de541efecbc None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:23:00.139428 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-fc777926-fd1b-47d3-b3e8-5de541efecbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:23:00.154275 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc777926-fd1b-47d3-b3e8-5de541efecbc None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:23:00.160448 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160c1141-0b62-4dc1-a431-acc66667f75e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:23:00.161118 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160c1141-0b62-4dc1-a431-acc66667f75e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:23:00.168462 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160c1141-0b62-4dc1-a431-acc66667f75e admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:23:00.168462 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160c1141-0b62-4dc1-a431-acc66667f75e admin admin] PATH_INFO: `/v3/projects/c16116bce0b943bdba7d985219bb18e6` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:23:00.168462 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160c1141-0b62-4dc1-a431-acc66667f75e admin admin] RBAC: Authorizing `identity:delete_project(project_id=c16116bce0b943bdba7d985219bb18e6)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:23:00.168462 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160c1141-0b62-4dc1-a431-acc66667f75e admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:23:00.322610 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-160c1141-0b62-4dc1-a431-acc66667f75e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c16116bce0b943bdba7d985219bb18e6', 'request_id': 'req-160c1141-0b62-4dc1-a431-acc66667f75e'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:23:00.328409 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-160c1141-0b62-4dc1-a431-acc66667f75e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c16116bce0b943bdba7d985219bb18e6', 'request_id': 'req-160c1141-0b62-4dc1-a431-acc66667f75e'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:23:00.335106 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-160c1141-0b62-4dc1-a431-acc66667f75e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c16116bce0b943bdba7d985219bb18e6', 'request_id': 'req-160c1141-0b62-4dc1-a431-acc66667f75e'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:23:00.441026 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-160c1141-0b62-4dc1-a431-acc66667f75e admin admin] The token cache is being invalidate because project c16116bce0b943bdba7d985219bb18e6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81677) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 10 12:23:00.441330 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-160c1141-0b62-4dc1-a431-acc66667f75e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:23:00.442044 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-160c1141-0b62-4dc1-a431-acc66667f75e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:23:00.443635 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1452/2882] 10.222.0.46 () {58 vars in 1233 bytes} [Mon Oct 10 12:23:00 2022] DELETE /identity/v3/projects/c16116bce0b943bdba7d985219bb18e6 => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:23:00.448941 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b54be87-47cc-4b15-be7d-396642b5ff06 None None] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:23:00.449383 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b54be87-47cc-4b15-be7d-396642b5ff06 None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:23:00.449739 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b54be87-47cc-4b15-be7d-396642b5ff06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:23:00.472489 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.auth.core [None req-8b54be87-47cc-4b15-be7d-396642b5ff06 None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:23:00.473725 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-8b54be87-47cc-4b15-be7d-396642b5ff06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:23:00.541279 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1431/2883] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:23:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:23:00.549910 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1235b07f-2a64-4dfa-a675-3ae540400717 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:23:00.550305 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1235b07f-2a64-4dfa-a675-3ae540400717 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:23:00.550476 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1235b07f-2a64-4dfa-a675-3ae540400717 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:23:00.559541 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:23:00.573312 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-1235b07f-2a64-4dfa-a675-3ae540400717 None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:23:00.574523 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-1235b07f-2a64-4dfa-a675-3ae540400717 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:23:00.583083 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1453/2884] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:23:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:23:00.587942 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b54be87-47cc-4b15-be7d-396642b5ff06 None None] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:23:00.598951 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b54be87-47cc-4b15-be7d-396642b5ff06 None None] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:23:00.603077 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1235b07f-2a64-4dfa-a675-3ae540400717 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:23:00.604963 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535fbcde-3521-4505-a26f-e30a36b2a5a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:23:00.605543 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-1235b07f-2a64-4dfa-a675-3ae540400717 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:23:00.605736 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535fbcde-3521-4505-a26f-e30a36b2a5a5 admin admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:23:00.605908 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535fbcde-3521-4505-a26f-e30a36b2a5a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:23:00.606128 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535fbcde-3521-4505-a26f-e30a36b2a5a5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:23:00.606847 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535fbcde-3521-4505-a26f-e30a36b2a5a5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81676) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 12:23:00.607155 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535fbcde-3521-4505-a26f-e30a36b2a5a5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:23:00.608876 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535fbcde-3521-4505-a26f-e30a36b2a5a5 admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:23:00.625944 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1432/2885] 10.222.0.46 () {58 vars in 1153 bytes} [Mon Oct 10 12:23:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 10 12:23:00.627402 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1235b07f-2a64-4dfa-a675-3ae540400717 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:23:00.630740 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535fbcde-3521-4505-a26f-e30a36b2a5a5 admin admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:23:00.631828 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ec3c32-7591-48d4-ab38-f900367721bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:23:00.632833 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ec3c32-7591-48d4-ab38-f900367721bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:23:00.633158 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ec3c32-7591-48d4-ab38-f900367721bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:23:00.633449 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ec3c32-7591-48d4-ab38-f900367721bf admin admin] PATH_INFO: `/v3/projects/d631ef3b88d440f1819d18da7117ee21` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:23:00.634661 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ec3c32-7591-48d4-ab38-f900367721bf admin admin] RBAC: Authorizing `identity:delete_project(project_id=d631ef3b88d440f1819d18da7117ee21)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:23:00.636624 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ec3c32-7591-48d4-ab38-f900367721bf admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:23:00.643280 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535fbcde-3521-4505-a26f-e30a36b2a5a5 admin admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:23:00.649462 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4fbe73-eb5f-4602-b86d-4b2c213eda12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:23:00.650091 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4fbe73-eb5f-4602-b86d-4b2c213eda12 admin admin] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:23:00.650525 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4fbe73-eb5f-4602-b86d-4b2c213eda12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:23:00.650882 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4fbe73-eb5f-4602-b86d-4b2c213eda12 admin admin] PATH_INFO: `/v3/projects` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:23:00.657909 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4fbe73-eb5f-4602-b86d-4b2c213eda12 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:23:00.657909 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4fbe73-eb5f-4602-b86d-4b2c213eda12 admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:23:00.767694 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-57ec3c32-7591-48d4-ab38-f900367721bf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd631ef3b88d440f1819d18da7117ee21', 'request_id': 'req-57ec3c32-7591-48d4-ab38-f900367721bf'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:23:00.773391 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-57ec3c32-7591-48d4-ab38-f900367721bf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd631ef3b88d440f1819d18da7117ee21', 'request_id': 'req-57ec3c32-7591-48d4-ab38-f900367721bf'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:23:00.779547 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-57ec3c32-7591-48d4-ab38-f900367721bf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd631ef3b88d440f1819d18da7117ee21', 'request_id': 'req-57ec3c32-7591-48d4-ab38-f900367721bf'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:23:00.792303 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1433/2886] 10.222.0.46 () {60 vars in 1153 bytes} [Mon Oct 10 12:23:00 2022] POST /identity/v3/projects => generated 361 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 12:23:00.793464 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4fbe73-eb5f-4602-b86d-4b2c213eda12 admin admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:23:00.859057 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4fbe73-eb5f-4602-b86d-4b2c213eda12 admin admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:23:00.864682 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a291d60c-cdf6-4c03-9512-f4601bfa7d57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:23:00.865331 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a291d60c-cdf6-4c03-9512-f4601bfa7d57 admin admin] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:23:00.865614 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a291d60c-cdf6-4c03-9512-f4601bfa7d57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:23:00.865887 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a291d60c-cdf6-4c03-9512-f4601bfa7d57 admin admin] PATH_INFO: `/v3/users` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:23:00.866532 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a291d60c-cdf6-4c03-9512-f4601bfa7d57 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:23:00.868556 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a291d60c-cdf6-4c03-9512-f4601bfa7d57 admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:23:00.891120 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-57ec3c32-7591-48d4-ab38-f900367721bf admin admin] The token cache is being invalidate because project d631ef3b88d440f1819d18da7117ee21 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81677) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 10 12:23:00.891419 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-57ec3c32-7591-48d4-ab38-f900367721bf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:23:00.892810 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-57ec3c32-7591-48d4-ab38-f900367721bf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:23:00.893689 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1454/2887] 10.222.0.46 () {58 vars in 1233 bytes} [Mon Oct 10 12:23:00 2022] DELETE /identity/v3/projects/d631ef3b88d440f1819d18da7117ee21 => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:23:00.899031 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb3a5c3-b062-485b-9d26-6391aef2512d None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:23:00.899313 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb3a5c3-b062-485b-9d26-6391aef2512d None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:23:00.899589 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb3a5c3-b062-485b-9d26-6391aef2512d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:23:00.920964 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-0cb3a5c3-b062-485b-9d26-6391aef2512d None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:23:00.922976 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-0cb3a5c3-b062-485b-9d26-6391aef2512d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:23:00.931332 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1455/2888] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:23:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 10 12:23:00.941612 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cf7235-879b-4a78-bb3f-a3d7ca7ee669 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:23:00.942262 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cf7235-879b-4a78-bb3f-a3d7ca7ee669 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:23:00.942262 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cf7235-879b-4a78-bb3f-a3d7ca7ee669 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:23:00.975484 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-c7cf7235-879b-4a78-bb3f-a3d7ca7ee669 None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:23:00.976703 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-c7cf7235-879b-4a78-bb3f-a3d7ca7ee669 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:23:00.985212 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1456/2889] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:23:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 10 12:23:00.989671 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7cf7235-879b-4a78-bb3f-a3d7ca7ee669 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:23:01.000714 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1434/2890] 10.222.0.46 () {60 vars in 1144 bytes} [Mon Oct 10 12:23:00 2022] POST /identity/v3/users => generated 281 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 10 12:23:01.001005 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7cf7235-879b-4a78-bb3f-a3d7ca7ee669 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:23:01.001968 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a291d60c-cdf6-4c03-9512-f4601bfa7d57 admin admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:23:01.003679 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-a291d60c-cdf6-4c03-9512-f4601bfa7d57 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:23:01.005815 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0102557b-9f62-4e74-b3b3-1787a56f3032 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:23:01.006468 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0102557b-9f62-4e74-b3b3-1787a56f3032 admin admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:23:01.006784 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0102557b-9f62-4e74-b3b3-1787a56f3032 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:23:01.007074 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0102557b-9f62-4e74-b3b3-1787a56f3032 admin admin] PATH_INFO: `/v3/domains` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:23:01.007828 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0102557b-9f62-4e74-b3b3-1787a56f3032 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81677) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 10 12:23:01.008696 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0102557b-9f62-4e74-b3b3-1787a56f3032 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:23:01.009711 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0102557b-9f62-4e74-b3b3-1787a56f3032 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:23:01.017834 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a291d60c-cdf6-4c03-9512-f4601bfa7d57 admin admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:23:01.025153 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1457/2891] 10.222.0.46 () {58 vars in 1153 bytes} [Mon Oct 10 12:23:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 10 12:23:01.025305 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d595656c-def9-4a3c-9425-a7d9735d4eda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:23:01.025305 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d595656c-def9-4a3c-9425-a7d9735d4eda admin admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:23:01.025305 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d595656c-def9-4a3c-9425-a7d9735d4eda admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:23:01.025613 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d595656c-def9-4a3c-9425-a7d9735d4eda admin admin] PATH_INFO: `/v3/roles` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:23:01.026174 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d595656c-def9-4a3c-9425-a7d9735d4eda admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:23:01.027857 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d595656c-def9-4a3c-9425-a7d9735d4eda admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:23:01.028248 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0102557b-9f62-4e74-b3b3-1787a56f3032 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:23:01.043793 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1435/2892] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 10 12:23:01 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 12:23:01.044018 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0102557b-9f62-4e74-b3b3-1787a56f3032 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:23:01.044898 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d595656c-def9-4a3c-9425-a7d9735d4eda admin admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:23:01.047556 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76d05d7-6f59-4c7e-98b7-05b7aebf3289 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:23:01.048166 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76d05d7-6f59-4c7e-98b7-05b7aebf3289 admin admin] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:23:01.048445 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76d05d7-6f59-4c7e-98b7-05b7aebf3289 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:23:01.048721 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76d05d7-6f59-4c7e-98b7-05b7aebf3289 admin admin] PATH_INFO: `/v3/projects` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:23:01.049353 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76d05d7-6f59-4c7e-98b7-05b7aebf3289 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:23:01.051326 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76d05d7-6f59-4c7e-98b7-05b7aebf3289 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:23:01.054779 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d595656c-def9-4a3c-9425-a7d9735d4eda admin admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:23:01.059568 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4c1ca1-544d-4881-9727-9fcc22c55b8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:23:01.060009 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4c1ca1-544d-4881-9727-9fcc22c55b8b admin admin] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:23:01.060287 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4c1ca1-544d-4881-9727-9fcc22c55b8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:23:01.060559 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4c1ca1-544d-4881-9727-9fcc22c55b8b admin admin] PATH_INFO: `/v3/roles` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:23:01.061199 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4c1ca1-544d-4881-9727-9fcc22c55b8b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:23:01.062842 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4c1ca1-544d-4881-9727-9fcc22c55b8b admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:23:01.074212 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1436/2893] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 10 12:23:01 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 12:23:01.079253 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4c1ca1-544d-4881-9727-9fcc22c55b8b admin admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:23:01.090544 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4c1ca1-544d-4881-9727-9fcc22c55b8b admin admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:23:01.096043 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71929106-d5f1-4768-a656-84bc81612057 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:23:01.096711 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71929106-d5f1-4768-a656-84bc81612057 admin admin] REQUEST_METHOD: `PUT` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:23:01.096994 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71929106-d5f1-4768-a656-84bc81612057 admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:23:01.097275 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71929106-d5f1-4768-a656-84bc81612057 admin admin] PATH_INFO: `/v3/projects/4cac127d59f94678a732fc232c56b05c/users/6f07c23a13a74988b5658f9ae929e4cb/roles/b2cb2a6e818e47338ce071e6fedfc673` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:23:01.110452 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:23:01.113838 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71929106-d5f1-4768-a656-84bc81612057 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4cac127d59f94678a732fc232c56b05c, user_id=6f07c23a13a74988b5658f9ae929e4cb, role_id=b2cb2a6e818e47338ce071e6fedfc673)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:23:01.118046 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71929106-d5f1-4768-a656-84bc81612057 admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:23:01.179156 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1458/2894] 10.222.0.46 () {60 vars in 1153 bytes} [Mon Oct 10 12:23:01 2022] POST /identity/v3/projects => generated 381 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 10 12:23:01.183173 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76d05d7-6f59-4c7e-98b7-05b7aebf3289 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:23:01.220931 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1437/2895] 10.222.0.46 () {60 vars in 1483 bytes} [Mon Oct 10 12:23:01 2022] PUT /identity/v3/projects/4cac127d59f94678a732fc232c56b05c/users/6f07c23a13a74988b5658f9ae929e4cb/roles/b2cb2a6e818e47338ce071e6fedfc673 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 10 12:23:01.235529 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27052360-6ec7-467e-beb2-ccb788349dc0 None None] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:23:01.235836 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27052360-6ec7-467e-beb2-ccb788349dc0 None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:23:01.236112 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27052360-6ec7-467e-beb2-ccb788349dc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:23:01.252665 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76d05d7-6f59-4c7e-98b7-05b7aebf3289 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:23:01.271867 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.auth.core [None req-27052360-6ec7-467e-beb2-ccb788349dc0 None None] MFA Rules not processed for user `6f07c23a13a74988b5658f9ae929e4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=81676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:23:01.277497 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-27052360-6ec7-467e-beb2-ccb788349dc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:23:01.295362 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:23:01.323100 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec858c3-7197-4fc7-9eb8-5c6b51f8c4d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:23:01.324048 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec858c3-7197-4fc7-9eb8-5c6b51f8c4d2 admin admin] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:23:01.324418 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec858c3-7197-4fc7-9eb8-5c6b51f8c4d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:23:01.324908 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec858c3-7197-4fc7-9eb8-5c6b51f8c4d2 admin admin] PATH_INFO: `/v3/users` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:23:01.325628 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec858c3-7197-4fc7-9eb8-5c6b51f8c4d2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:23:01.327802 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec858c3-7197-4fc7-9eb8-5c6b51f8c4d2 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:23:01.357999 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1438/2896] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:23:01 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:23:01.400847 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27052360-6ec7-467e-beb2-ccb788349dc0 None None] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:23:01.400847 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-27052360-6ec7-467e-beb2-ccb788349dc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:23:01.462600 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1459/2897] 10.222.0.46 () {60 vars in 1144 bytes} [Mon Oct 10 12:23:01 2022] POST /identity/v3/users => generated 291 bytes in 275 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 10 12:23:01.464079 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec858c3-7197-4fc7-9eb8-5c6b51f8c4d2 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:23:01.470383 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27052360-6ec7-467e-beb2-ccb788349dc0 None None] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:23:01.474775 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da5f631-1aba-4bff-80b1-e787205a7d3c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3ae093470484b93b23dcfae1657df9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad3772f7b6a47c28b7ba37a9316a3c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:23:01.474946 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec858c3-7197-4fc7-9eb8-5c6b51f8c4d2 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:23:01.475721 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da5f631-1aba-4bff-80b1-e787205a7d3c service cinder] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:23:01.476105 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da5f631-1aba-4bff-80b1-e787205a7d3c service cinder] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:23:01.476514 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da5f631-1aba-4bff-80b1-e787205a7d3c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:23:01.484218 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813bf5aa-d368-4c4b-9220-6ac7990d4909 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:23:01.484883 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813bf5aa-d368-4c4b-9220-6ac7990d4909 admin admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:23:01.485168 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813bf5aa-d368-4c4b-9220-6ac7990d4909 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:23:01.485450 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813bf5aa-d368-4c4b-9220-6ac7990d4909 admin admin] PATH_INFO: `/v3/roles` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:23:01.486107 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813bf5aa-d368-4c4b-9220-6ac7990d4909 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:23:01.488018 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813bf5aa-d368-4c4b-9220-6ac7990d4909 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:23:01.491947 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da5f631-1aba-4bff-80b1-e787205a7d3c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:23:01.493457 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da5f631-1aba-4bff-80b1-e787205a7d3c service cinder] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:23:01.503176 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1460/2898] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 10 12:23:01 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 10 12:23:01.504777 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813bf5aa-d368-4c4b-9220-6ac7990d4909 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:23:01.505296 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1439/2899] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 10 12:23:01 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 10 12:23:01.516712 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813bf5aa-d368-4c4b-9220-6ac7990d4909 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:23:01.524564 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da5f631-1aba-4bff-80b1-e787205a7d3c service cinder] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:23:01.526500 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45042b1e-84ef-42c2-95a2-bd5a60041fe5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:23:01.527367 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45042b1e-84ef-42c2-95a2-bd5a60041fe5 admin admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:23:01.527876 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45042b1e-84ef-42c2-95a2-bd5a60041fe5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:23:01.528152 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45042b1e-84ef-42c2-95a2-bd5a60041fe5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:23:01.528886 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45042b1e-84ef-42c2-95a2-bd5a60041fe5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:23:01.530550 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45042b1e-84ef-42c2-95a2-bd5a60041fe5 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:23:01.535435 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da5f631-1aba-4bff-80b1-e787205a7d3c service cinder] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:23:01.540065 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1d354b-dee7-4e25-aad8-77167a665bc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:23:01.544238 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1d354b-dee7-4e25-aad8-77167a665bc2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:23:01.544238 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1d354b-dee7-4e25-aad8-77167a665bc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:23:01.544238 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1d354b-dee7-4e25-aad8-77167a665bc2 admin admin] PATH_INFO: `/v3/users/6f07c23a13a74988b5658f9ae929e4cb` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:23:01.544238 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1d354b-dee7-4e25-aad8-77167a665bc2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6f07c23a13a74988b5658f9ae929e4cb)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:23:01.544625 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1461/2900] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 10 12:23:01 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 10 12:23:01.544727 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1d354b-dee7-4e25-aad8-77167a665bc2 admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:23:01.547238 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45042b1e-84ef-42c2-95a2-bd5a60041fe5 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:23:01.558706 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45042b1e-84ef-42c2-95a2-bd5a60041fe5 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:23:01.564019 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385c32e5-30e3-4ac5-baad-963ae1509eb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:23:01.564694 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385c32e5-30e3-4ac5-baad-963ae1509eb1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:23:01.564981 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385c32e5-30e3-4ac5-baad-963ae1509eb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:23:01.565263 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385c32e5-30e3-4ac5-baad-963ae1509eb1 admin admin] PATH_INFO: `/v3/projects/bafe96e8ab4d498981d0e309c484c125/users/2db334d548cf45a7a338a7fcc30ac3c4/roles/b2cb2a6e818e47338ce071e6fedfc673` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:23:01.584987 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385c32e5-30e3-4ac5-baad-963ae1509eb1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bafe96e8ab4d498981d0e309c484c125, user_id=2db334d548cf45a7a338a7fcc30ac3c4, role_id=b2cb2a6e818e47338ce071e6fedfc673)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:23:01.587854 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385c32e5-30e3-4ac5-baad-963ae1509eb1 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:23:01.792711 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1462/2901] 10.222.0.46 () {60 vars in 1483 bytes} [Mon Oct 10 12:23:01 2022] PUT /identity/v3/projects/bafe96e8ab4d498981d0e309c484c125/users/2db334d548cf45a7a338a7fcc30ac3c4/roles/b2cb2a6e818e47338ce071e6fedfc673 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 10 12:23:01.798454 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12a5706-e620-4ddf-8b8e-3eeaf65594e1 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:23:01.798918 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12a5706-e620-4ddf-8b8e-3eeaf65594e1 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:23:01.799205 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12a5706-e620-4ddf-8b8e-3eeaf65594e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:23:01.820120 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-f12a5706-e620-4ddf-8b8e-3eeaf65594e1 None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:23:01.821364 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-f12a5706-e620-4ddf-8b8e-3eeaf65594e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:23:01.891170 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-da1d354b-dee7-4e25-aad8-77167a665bc2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6f07c23a13a74988b5658f9ae929e4cb'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:23:01.892996 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1463/2902] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:23:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 10 12:23:01.903625 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12a5706-e620-4ddf-8b8e-3eeaf65594e1 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:23:01.905223 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-f12a5706-e620-4ddf-8b8e-3eeaf65594e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:23:01.908708 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-da1d354b-dee7-4e25-aad8-77167a665bc2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6f07c23a13a74988b5658f9ae929e4cb'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:23:01.909416 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-da1d354b-dee7-4e25-aad8-77167a665bc2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6f07c23a13a74988b5658f9ae929e4cb'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:23:01.969435 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12a5706-e620-4ddf-8b8e-3eeaf65594e1 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:23:01.973301 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce64634-7ded-407d-b106-ef2b688caf6d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58b1c7c85302421cba823ace4201861f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad3772f7b6a47c28b7ba37a9316a3c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:23:01.973916 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce64634-7ded-407d-b106-ef2b688caf6d service neutron] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:23:01.975847 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce64634-7ded-407d-b106-ef2b688caf6d service neutron] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:23:01.975847 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce64634-7ded-407d-b106-ef2b688caf6d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:23:01.976255 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-9ce64634-7ded-407d-b106-ef2b688caf6d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:23:01.989018 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce64634-7ded-407d-b106-ef2b688caf6d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:23:01.990439 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce64634-7ded-407d-b106-ef2b688caf6d service neutron] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:23:01.994384 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-da1d354b-dee7-4e25-aad8-77167a665bc2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6f07c23a13a74988b5658f9ae929e4cb'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:23:02.000983 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1464/2903] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 10 12:23:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:23:02.008772 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:23:02.010287 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-da1d354b-dee7-4e25-aad8-77167a665bc2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6f07c23a13a74988b5658f9ae929e4cb'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:23:02.018115 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1440/2904] 10.222.0.46 () {58 vars in 1224 bytes} [Mon Oct 10 12:23:01 2022] DELETE /identity/v3/users/6f07c23a13a74988b5658f9ae929e4cb => generated 0 bytes in 494 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 10 12:23:02.022449 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf26955-c554-4ca2-9e24-506331648edc None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:23:02.022854 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf26955-c554-4ca2-9e24-506331648edc None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:23:02.023228 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf26955-c554-4ca2-9e24-506331648edc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:23:02.047746 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-fcf26955-c554-4ca2-9e24-506331648edc None None] MFA Rules not processed for user `8cbbc9233ced43d487079e9479dc555c`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:23:02.048895 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-fcf26955-c554-4ca2-9e24-506331648edc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:23:02.070453 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:23:02.113001 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1465/2905] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:23:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:23:02.122893 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1d354b-dee7-4e25-aad8-77167a665bc2 admin admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:23:02.124424 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-da1d354b-dee7-4e25-aad8-77167a665bc2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:23:02.197328 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1d354b-dee7-4e25-aad8-77167a665bc2 admin admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:23:02.200990 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52cf449a-bcb8-48b0-b955-0caa8bf65d01 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58b1c7c85302421cba823ace4201861f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad3772f7b6a47c28b7ba37a9316a3c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:23:02.201692 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cf449a-bcb8-48b0-b955-0caa8bf65d01 service neutron] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:23:02.201880 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cf449a-bcb8-48b0-b955-0caa8bf65d01 service neutron] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:23:02.202151 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cf449a-bcb8-48b0-b955-0caa8bf65d01 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:23:02.212665 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52cf449a-bcb8-48b0-b955-0caa8bf65d01 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:23:02.214137 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52cf449a-bcb8-48b0-b955-0caa8bf65d01 service neutron] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:23:02.224389 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1441/2906] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 10 12:23:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:23:02.785307 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf26955-c554-4ca2-9e24-506331648edc None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:23:02.786794 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-fcf26955-c554-4ca2-9e24-506331648edc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:23:02.801636 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf26955-c554-4ca2-9e24-506331648edc None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:23:02.814577 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8012e9e0-aa17-4874-b21b-7b8e2a05d71c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:23:02.815313 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8012e9e0-aa17-4874-b21b-7b8e2a05d71c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:23:02.815492 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8012e9e0-aa17-4874-b21b-7b8e2a05d71c admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:23:02.815805 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8012e9e0-aa17-4874-b21b-7b8e2a05d71c admin admin] PATH_INFO: `/v3/projects/4cac127d59f94678a732fc232c56b05c` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:23:02.816952 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8012e9e0-aa17-4874-b21b-7b8e2a05d71c admin admin] RBAC: Authorizing `identity:delete_project(project_id=4cac127d59f94678a732fc232c56b05c)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:23:02.822101 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8012e9e0-aa17-4874-b21b-7b8e2a05d71c admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:23:02.950818 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-8012e9e0-aa17-4874-b21b-7b8e2a05d71c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4cac127d59f94678a732fc232c56b05c', 'request_id': 'req-8012e9e0-aa17-4874-b21b-7b8e2a05d71c'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:23:02.960039 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-8012e9e0-aa17-4874-b21b-7b8e2a05d71c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4cac127d59f94678a732fc232c56b05c', 'request_id': 'req-8012e9e0-aa17-4874-b21b-7b8e2a05d71c'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:23:02.966818 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-8012e9e0-aa17-4874-b21b-7b8e2a05d71c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4cac127d59f94678a732fc232c56b05c', 'request_id': 'req-8012e9e0-aa17-4874-b21b-7b8e2a05d71c'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:23:03.058333 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-8012e9e0-aa17-4874-b21b-7b8e2a05d71c admin admin] The token cache is being invalidate because project 4cac127d59f94678a732fc232c56b05c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81677) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 10 12:23:03.059480 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-8012e9e0-aa17-4874-b21b-7b8e2a05d71c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:23:03.060397 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-8012e9e0-aa17-4874-b21b-7b8e2a05d71c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:23:03.062075 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1466/2907] 10.222.0.46 () {58 vars in 1233 bytes} [Mon Oct 10 12:23:02 2022] DELETE /identity/v3/projects/4cac127d59f94678a732fc232c56b05c => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:23:08.179655 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aba44ff-415c-43a1-a64e-cadcf5f1fb2e None None] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:23:08.180240 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aba44ff-415c-43a1-a64e-cadcf5f1fb2e None None] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:23:08.180240 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aba44ff-415c-43a1-a64e-cadcf5f1fb2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:23:08.212608 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.auth.core [None req-4aba44ff-415c-43a1-a64e-cadcf5f1fb2e None None] MFA Rules not processed for user `2db334d548cf45a7a338a7fcc30ac3c4`. Rule list: `[]` (Enabled: `True`). {{(pid=81676) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:23:08.213837 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-4aba44ff-415c-43a1-a64e-cadcf5f1fb2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:23:08.304355 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1442/2908] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:23:08 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:23:08.304481 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8012e9e0-aa17-4874-b21b-7b8e2a05d71c admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:23:08.304481 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-8012e9e0-aa17-4874-b21b-7b8e2a05d71c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:23:08.376162 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8012e9e0-aa17-4874-b21b-7b8e2a05d71c admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:23:08.380794 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac16c818-15dc-4c2e-8314-d3f21b3b22f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:23:08.381403 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac16c818-15dc-4c2e-8314-d3f21b3b22f2 admin admin] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:23:08.381679 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac16c818-15dc-4c2e-8314-d3f21b3b22f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:23:08.381946 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac16c818-15dc-4c2e-8314-d3f21b3b22f2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:23:08.382574 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac16c818-15dc-4c2e-8314-d3f21b3b22f2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:23:08.384512 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac16c818-15dc-4c2e-8314-d3f21b3b22f2 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:23:08.478730 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1467/2909] 10.222.0.46 () {60 vars in 1153 bytes} [Mon Oct 10 12:23:08 2022] POST /identity/v3/projects => generated 385 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 12:23:08.482694 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aba44ff-415c-43a1-a64e-cadcf5f1fb2e None None] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:23:08.494188 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aba44ff-415c-43a1-a64e-cadcf5f1fb2e None None] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:23:08.558209 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1759fb1-bb35-49ed-bc3c-d9618074066d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:23:08.558948 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1759fb1-bb35-49ed-bc3c-d9618074066d admin admin] REQUEST_METHOD: `POST` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:23:08.559236 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1759fb1-bb35-49ed-bc3c-d9618074066d admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:23:08.559510 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1759fb1-bb35-49ed-bc3c-d9618074066d admin admin] PATH_INFO: `/v3/users` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:23:08.560154 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1759fb1-bb35-49ed-bc3c-d9618074066d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:23:08.562131 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1759fb1-bb35-49ed-bc3c-d9618074066d admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:23:08.644163 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1443/2910] 10.222.0.46 () {60 vars in 1144 bytes} [Mon Oct 10 12:23:08 2022] POST /identity/v3/users => generated 299 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 10 12:23:08.648391 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac16c818-15dc-4c2e-8314-d3f21b3b22f2 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:23:08.664873 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac16c818-15dc-4c2e-8314-d3f21b3b22f2 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:23:08.670512 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093c6527-1c77-449e-9888-d8cd305edafa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:23:08.672077 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093c6527-1c77-449e-9888-d8cd305edafa admin admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:23:08.672556 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093c6527-1c77-449e-9888-d8cd305edafa admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:23:08.672959 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093c6527-1c77-449e-9888-d8cd305edafa admin admin] PATH_INFO: `/v3/roles` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:23:08.673812 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093c6527-1c77-449e-9888-d8cd305edafa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:23:08.676412 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093c6527-1c77-449e-9888-d8cd305edafa admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:23:08.689287 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1468/2911] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 10 12:23:08 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 12:23:08.694234 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1759fb1-bb35-49ed-bc3c-d9618074066d admin admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:23:08.705597 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1759fb1-bb35-49ed-bc3c-d9618074066d admin admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:23:08.711996 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0e6c03-e863-4baf-93df-0ef55acded86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:23:08.712709 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0e6c03-e863-4baf-93df-0ef55acded86 admin admin] REQUEST_METHOD: `PUT` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:23:08.713030 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0e6c03-e863-4baf-93df-0ef55acded86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:23:08.713308 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0e6c03-e863-4baf-93df-0ef55acded86 admin admin] PATH_INFO: `/v3/projects/91f85d93de9b44dc9ef044e3f7ecb7c4/users/aae29306bca048dab6445e5db59a8a95/roles/0d822a0674b6495fac623877a3b38025` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:23:08.730152 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0e6c03-e863-4baf-93df-0ef55acded86 admin admin] RBAC: Authorizing `identity:create_grant(project_id=91f85d93de9b44dc9ef044e3f7ecb7c4, user_id=aae29306bca048dab6445e5db59a8a95, role_id=0d822a0674b6495fac623877a3b38025)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:23:08.733093 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0e6c03-e863-4baf-93df-0ef55acded86 admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:23:08.736469 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:23:08.838002 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1444/2912] 10.222.0.46 () {60 vars in 1483 bytes} [Mon Oct 10 12:23:08 2022] PUT /identity/v3/projects/91f85d93de9b44dc9ef044e3f7ecb7c4/users/aae29306bca048dab6445e5db59a8a95/roles/0d822a0674b6495fac623877a3b38025 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:23:08.842748 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093c6527-1c77-449e-9888-d8cd305edafa admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:23:08.854122 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093c6527-1c77-449e-9888-d8cd305edafa admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:23:08.884302 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:23:08.940745 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e08ed3c-379b-4089-8067-4a2be2e3a32a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:23:08.942010 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e08ed3c-379b-4089-8067-4a2be2e3a32a admin admin] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:23:08.942559 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e08ed3c-379b-4089-8067-4a2be2e3a32a admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:23:08.943012 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e08ed3c-379b-4089-8067-4a2be2e3a32a admin admin] PATH_INFO: `/v3/roles` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:23:08.944193 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e08ed3c-379b-4089-8067-4a2be2e3a32a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:23:08.947476 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e08ed3c-379b-4089-8067-4a2be2e3a32a admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:23:08.972862 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1469/2913] 10.222.0.46 () {58 vars in 1122 bytes} [Mon Oct 10 12:23:08 2022] GET /identity/v3/roles => generated 1420 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 10 12:23:08.979145 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0e6c03-e863-4baf-93df-0ef55acded86 admin admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:23:09.013975 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0e6c03-e863-4baf-93df-0ef55acded86 admin admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:23:09.022014 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a1e7f1-4e4d-4c13-a0b6-7ef37b943cf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:23:09.023159 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a1e7f1-4e4d-4c13-a0b6-7ef37b943cf1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:23:09.023661 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a1e7f1-4e4d-4c13-a0b6-7ef37b943cf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:23:09.024157 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a1e7f1-4e4d-4c13-a0b6-7ef37b943cf1 admin admin] PATH_INFO: `/v3/projects/91f85d93de9b44dc9ef044e3f7ecb7c4/users/aae29306bca048dab6445e5db59a8a95/roles/0d822a0674b6495fac623877a3b38025` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:23:09.027819 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a1e7f1-4e4d-4c13-a0b6-7ef37b943cf1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=91f85d93de9b44dc9ef044e3f7ecb7c4, user_id=aae29306bca048dab6445e5db59a8a95, role_id=0d822a0674b6495fac623877a3b38025)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:23:09.033592 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a1e7f1-4e4d-4c13-a0b6-7ef37b943cf1 admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:23:09.056236 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1445/2914] 10.222.0.46 () {60 vars in 1483 bytes} [Mon Oct 10 12:23:08 2022] PUT /identity/v3/projects/91f85d93de9b44dc9ef044e3f7ecb7c4/users/aae29306bca048dab6445e5db59a8a95/roles/0d822a0674b6495fac623877a3b38025 => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:23:14.849142 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4cc764-e345-4b17-84f2-0d4e054dd056 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:23:14.849611 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4cc764-e345-4b17-84f2-0d4e054dd056 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:23:14.849750 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4cc764-e345-4b17-84f2-0d4e054dd056 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:23:14.882441 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-aa4cc764-e345-4b17-84f2-0d4e054dd056 None None] MFA Rules not processed for user `aae29306bca048dab6445e5db59a8a95`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:23:14.884528 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-aa4cc764-e345-4b17-84f2-0d4e054dd056 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:23:14.974111 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1470/2915] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:23:14 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:23:14.987346 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a1e7f1-4e4d-4c13-a0b6-7ef37b943cf1 admin admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:23:14.989153 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-92a1e7f1-4e4d-4c13-a0b6-7ef37b943cf1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:23:15.081677 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a1e7f1-4e4d-4c13-a0b6-7ef37b943cf1 admin admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:23:15.085904 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e97def-4cc5-453b-b5f6-1c085fe379fd service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '124f4aa8ab244ca2aadf4f2614df51e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad3772f7b6a47c28b7ba37a9316a3c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:23:15.086536 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e97def-4cc5-453b-b5f6-1c085fe379fd service glance] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:23:15.086934 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e97def-4cc5-453b-b5f6-1c085fe379fd service glance] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:23:15.087213 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e97def-4cc5-453b-b5f6-1c085fe379fd service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:23:15.157083 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e97def-4cc5-453b-b5f6-1c085fe379fd service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:23:15.164223 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e97def-4cc5-453b-b5f6-1c085fe379fd service glance] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:23:15.177136 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1446/2916] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 10 12:23:14 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:23:15.542882 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d18248-c9e9-41a1-9f5e-538eb97ce929 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:23:15.543399 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d18248-c9e9-41a1-9f5e-538eb97ce929 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:23:15.543897 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d18248-c9e9-41a1-9f5e-538eb97ce929 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:23:15.584688 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-13d18248-c9e9-41a1-9f5e-538eb97ce929 None None] MFA Rules not processed for user `d6430593eecf45d8b891c3b55de4a944`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:23:15.586047 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-13d18248-c9e9-41a1-9f5e-538eb97ce929 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:23:15.655702 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1471/2917] 10.222.0.46 () {58 vars in 1009 bytes} [Mon Oct 10 12:23:15 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:23:15.668142 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e97def-4cc5-453b-b5f6-1c085fe379fd service glance] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:23:15.669651 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-12e97def-4cc5-453b-b5f6-1c085fe379fd service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:23:15.751547 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e97def-4cc5-453b-b5f6-1c085fe379fd service glance] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:23:15.756727 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b1d0ce-c013-4258-a97e-1119d97bae86 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15ef3d53ea484237adb0b3d64fdc5480', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad3772f7b6a47c28b7ba37a9316a3c1', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:23:15.757351 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b1d0ce-c013-4258-a97e-1119d97bae86 service swift] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:23:15.757636 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b1d0ce-c013-4258-a97e-1119d97bae86 service swift] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:23:15.757911 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b1d0ce-c013-4258-a97e-1119d97bae86 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:23:15.767543 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b1d0ce-c013-4258-a97e-1119d97bae86 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:23:15.769158 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b1d0ce-c013-4258-a97e-1119d97bae86 service swift] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:23:15.779124 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1447/2918] 10.222.0.46 () {60 vars in 1384 bytes} [Mon Oct 10 12:23:15 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 10 12:23:25.648661 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=77 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:23:25.775830 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:INSERT=2,keystone:SELECT=87 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:23:26.331517 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607be372-a41f-44c4-9e44-88fd2fa72fda None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:23:26.331797 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607be372-a41f-44c4-9e44-88fd2fa72fda None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:23:26.332066 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607be372-a41f-44c4-9e44-88fd2fa72fda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:23:26.351441 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-607be372-a41f-44c4-9e44-88fd2fa72fda None None] MFA Rules not processed for user `2db334d548cf45a7a338a7fcc30ac3c4`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:23:26.352804 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-607be372-a41f-44c4-9e44-88fd2fa72fda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:23:26.359941 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1472/2919] 10.222.0.46 () {58 vars in 958 bytes} [Mon Oct 10 12:23:26 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:23:26.615744 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b1d0ce-c013-4258-a97e-1119d97bae86 service swift] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:23:26.617155 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-47b1d0ce-c013-4258-a97e-1119d97bae86 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:23:26.673670 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b1d0ce-c013-4258-a97e-1119d97bae86 service swift] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:23:26.677188 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f063b743-0e97-4202-be84-3fab2ae92d0e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58b1c7c85302421cba823ace4201861f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad3772f7b6a47c28b7ba37a9316a3c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:23:26.677783 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f063b743-0e97-4202-be84-3fab2ae92d0e service neutron] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:23:26.678064 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f063b743-0e97-4202-be84-3fab2ae92d0e service neutron] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:23:26.678333 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f063b743-0e97-4202-be84-3fab2ae92d0e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:23:26.680085 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-f063b743-0e97-4202-be84-3fab2ae92d0e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:23:26.689942 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f063b743-0e97-4202-be84-3fab2ae92d0e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:23:26.691381 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f063b743-0e97-4202-be84-3fab2ae92d0e service neutron] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:23:26.699372 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1448/2920] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 10 12:23:26 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:23:30.162435 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4908e48b-ef9e-4c98-b3d3-194685b70a04 None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:23:30.163172 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4908e48b-ef9e-4c98-b3d3-194685b70a04 None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:23:30.163460 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4908e48b-ef9e-4c98-b3d3-194685b70a04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:23:30.184807 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-4908e48b-ef9e-4c98-b3d3-194685b70a04 None None] MFA Rules not processed for user `d6430593eecf45d8b891c3b55de4a944`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:23:30.186024 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-4908e48b-ef9e-4c98-b3d3-194685b70a04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:23:30.194094 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1473/2921] 10.222.0.46 () {58 vars in 1009 bytes} [Mon Oct 10 12:23:30 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:23:30.207178 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f063b743-0e97-4202-be84-3fab2ae92d0e service neutron] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:23:30.219590 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f063b743-0e97-4202-be84-3fab2ae92d0e service neutron] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:23:30.224494 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c49094a-a4f9-415e-950d-31316791ca24 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15ef3d53ea484237adb0b3d64fdc5480', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad3772f7b6a47c28b7ba37a9316a3c1', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:23:30.235141 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c49094a-a4f9-415e-950d-31316791ca24 service swift] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:23:30.235141 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c49094a-a4f9-415e-950d-31316791ca24 service swift] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:23:30.235141 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c49094a-a4f9-415e-950d-31316791ca24 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:23:30.235141 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c49094a-a4f9-415e-950d-31316791ca24 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:23:30.235548 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c49094a-a4f9-415e-950d-31316791ca24 service swift] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:23:30.243656 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1449/2922] 10.222.0.46 () {60 vars in 1384 bytes} [Mon Oct 10 12:23:30 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 10 12:23:30.686975 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4908e48b-ef9e-4c98-b3d3-194685b70a04 None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:23:30.703655 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4908e48b-ef9e-4c98-b3d3-194685b70a04 None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:23:30.708220 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a55ec77-8716-4ab4-8d6b-77262bf96570 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58b1c7c85302421cba823ace4201861f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad3772f7b6a47c28b7ba37a9316a3c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:23:30.709175 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a55ec77-8716-4ab4-8d6b-77262bf96570 service neutron] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:23:30.709526 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a55ec77-8716-4ab4-8d6b-77262bf96570 service neutron] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:23:30.709960 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a55ec77-8716-4ab4-8d6b-77262bf96570 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:23:30.712002 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-0a55ec77-8716-4ab4-8d6b-77262bf96570 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:23:30.814357 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a55ec77-8716-4ab4-8d6b-77262bf96570 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:23:30.815846 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a55ec77-8716-4ab4-8d6b-77262bf96570 service neutron] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:23:30.828510 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1474/2923] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 10 12:23:30 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:23:40.240073 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=32 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:23:40.823316 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=40 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:24:02.461422 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c49094a-a4f9-415e-950d-31316791ca24 service swift] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:24:02.463175 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-9c49094a-a4f9-415e-950d-31316791ca24 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:24:02.476576 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c49094a-a4f9-415e-950d-31316791ca24 service swift] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:24:02.480147 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ef56b0-346f-4e46-af70-1d0d3fcfc11c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58b1c7c85302421cba823ace4201861f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad3772f7b6a47c28b7ba37a9316a3c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:24:02.480749 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ef56b0-346f-4e46-af70-1d0d3fcfc11c service neutron] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:24:02.481027 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ef56b0-346f-4e46-af70-1d0d3fcfc11c service neutron] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:24:02.481295 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ef56b0-346f-4e46-af70-1d0d3fcfc11c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:24:02.483207 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-b9ef56b0-346f-4e46-af70-1d0d3fcfc11c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:24:02.493039 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ef56b0-346f-4e46-af70-1d0d3fcfc11c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:24:02.494442 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ef56b0-346f-4e46-af70-1d0d3fcfc11c service neutron] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:24:02.502345 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1450/2924] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 10 12:24:02 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:24:04.750095 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a55ec77-8716-4ab4-8d6b-77262bf96570 service neutron] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:24:04.751643 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-0a55ec77-8716-4ab4-8d6b-77262bf96570 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:24:04.769213 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a55ec77-8716-4ab4-8d6b-77262bf96570 service neutron] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:24:04.775486 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2df673-042b-48c0-8d9a-a87b698b0d62 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f130bbb69e546ce99b4e14de81d7e54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad3772f7b6a47c28b7ba37a9316a3c1', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:24:04.776104 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2df673-042b-48c0-8d9a-a87b698b0d62 service nova] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:24:04.776383 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2df673-042b-48c0-8d9a-a87b698b0d62 service nova] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:24:04.779340 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2df673-042b-48c0-8d9a-a87b698b0d62 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:24:04.779340 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-7e2df673-042b-48c0-8d9a-a87b698b0d62 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:24:04.792025 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2df673-042b-48c0-8d9a-a87b698b0d62 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:24:04.793625 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2df673-042b-48c0-8d9a-a87b698b0d62 service nova] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:24:04.804745 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1475/2925] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 10 12:24:04 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:24:06.329248 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ef56b0-346f-4e46-af70-1d0d3fcfc11c service neutron] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:24:06.338721 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ef56b0-346f-4e46-af70-1d0d3fcfc11c service neutron] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:24:06.342440 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9ce728-5201-434e-b48b-1e5290624819 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58b1c7c85302421cba823ace4201861f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad3772f7b6a47c28b7ba37a9316a3c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:24:06.343077 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9ce728-5201-434e-b48b-1e5290624819 service neutron] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:24:06.343364 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9ce728-5201-434e-b48b-1e5290624819 service neutron] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:24:06.343635 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9ce728-5201-434e-b48b-1e5290624819 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:24:06.356830 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9ce728-5201-434e-b48b-1e5290624819 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:24:06.358254 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9ce728-5201-434e-b48b-1e5290624819 service neutron] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:24:06.368731 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1451/2926] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 10 12:24:06 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:24:11.954519 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2df673-042b-48c0-8d9a-a87b698b0d62 service nova] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:24:11.966045 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2df673-042b-48c0-8d9a-a87b698b0d62 service nova] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:24:11.972410 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed1296a-abfa-4ec2-90ae-931b3d0ce3ec service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f130bbb69e546ce99b4e14de81d7e54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad3772f7b6a47c28b7ba37a9316a3c1', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:24:11.973330 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed1296a-abfa-4ec2-90ae-931b3d0ce3ec service nova] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:24:11.973615 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed1296a-abfa-4ec2-90ae-931b3d0ce3ec service nova] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:24:11.973973 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed1296a-abfa-4ec2-90ae-931b3d0ce3ec service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:24:11.976259 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-2ed1296a-abfa-4ec2-90ae-931b3d0ce3ec service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:24:11.989281 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed1296a-abfa-4ec2-90ae-931b3d0ce3ec service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:24:11.991909 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed1296a-abfa-4ec2-90ae-931b3d0ce3ec service nova] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:24:12.004115 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1476/2927] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 10 12:24:11 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:24:16.364026 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=12 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:24:21.999117 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=12 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:25:43.894532 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9ce728-5201-434e-b48b-1e5290624819 service neutron] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:25:43.930590 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9ce728-5201-434e-b48b-1e5290624819 service neutron] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:25:43.934419 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e867b7d-23c5-4c61-91eb-6b687e1019ab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58b1c7c85302421cba823ace4201861f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad3772f7b6a47c28b7ba37a9316a3c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:25:43.935619 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e867b7d-23c5-4c61-91eb-6b687e1019ab service neutron] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:25:43.935960 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e867b7d-23c5-4c61-91eb-6b687e1019ab service neutron] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:25:43.936249 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e867b7d-23c5-4c61-91eb-6b687e1019ab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:25:43.938048 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-8e867b7d-23c5-4c61-91eb-6b687e1019ab service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:25:44.019867 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e867b7d-23c5-4c61-91eb-6b687e1019ab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:25:44.021294 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e867b7d-23c5-4c61-91eb-6b687e1019ab service neutron] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:25:44.030462 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1452/2928] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 10 12:25:43 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:25:47.956130 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed1296a-abfa-4ec2-90ae-931b3d0ce3ec service nova] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:25:47.998740 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed1296a-abfa-4ec2-90ae-931b3d0ce3ec service nova] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:25:48.005565 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac885ea9-fe43-48a6-ac1d-4479740c124e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f130bbb69e546ce99b4e14de81d7e54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad3772f7b6a47c28b7ba37a9316a3c1', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:25:48.006211 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac885ea9-fe43-48a6-ac1d-4479740c124e service nova] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:25:48.006487 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac885ea9-fe43-48a6-ac1d-4479740c124e service nova] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:25:48.006795 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac885ea9-fe43-48a6-ac1d-4479740c124e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:25:48.008560 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-ac885ea9-fe43-48a6-ac1d-4479740c124e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:25:48.020872 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac885ea9-fe43-48a6-ac1d-4479740c124e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:25:48.022525 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac885ea9-fe43-48a6-ac1d-4479740c124e service nova] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:25:48.033269 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1477/2929] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 10 12:25:47 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 78 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:25:50.444650 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e867b7d-23c5-4c61-91eb-6b687e1019ab service neutron] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:25:50.454223 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e867b7d-23c5-4c61-91eb-6b687e1019ab service neutron] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:25:50.458421 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298eff13-61ba-46a8-a730-2f3343bce68d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '58b1c7c85302421cba823ace4201861f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad3772f7b6a47c28b7ba37a9316a3c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:25:50.459081 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298eff13-61ba-46a8-a730-2f3343bce68d service neutron] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:25:50.459364 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298eff13-61ba-46a8-a730-2f3343bce68d service neutron] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:25:50.459639 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298eff13-61ba-46a8-a730-2f3343bce68d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:25:50.461737 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-298eff13-61ba-46a8-a730-2f3343bce68d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:25:50.547226 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298eff13-61ba-46a8-a730-2f3343bce68d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:25:50.548652 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298eff13-61ba-46a8-a730-2f3343bce68d service neutron] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:25:50.569347 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1453/2930] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 10 12:25:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:25:57.503156 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac885ea9-fe43-48a6-ac1d-4479740c124e service nova] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:25:57.504580 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-ac885ea9-fe43-48a6-ac1d-4479740c124e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:25:57.600070 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac885ea9-fe43-48a6-ac1d-4479740c124e service nova] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:25:57.605108 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7a745c-ab1c-4c55-9d39-f8e5d100d509 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e8f9bb1776d47169ef7febb57c954cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad3772f7b6a47c28b7ba37a9316a3c1', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:25:57.605713 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7a745c-ab1c-4c55-9d39-f8e5d100d509 service placement] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:25:57.606041 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7a745c-ab1c-4c55-9d39-f8e5d100d509 service placement] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:25:57.606323 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7a745c-ab1c-4c55-9d39-f8e5d100d509 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:25:57.608108 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-8e7a745c-ab1c-4c55-9d39-f8e5d100d509 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:25:57.620651 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7a745c-ab1c-4c55-9d39-f8e5d100d509 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:25:57.622309 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7a745c-ab1c-4c55-9d39-f8e5d100d509 service placement] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:25:57.632538 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1478/2931] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 10 12:25:57 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:25:57.652205 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298eff13-61ba-46a8-a730-2f3343bce68d service neutron] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:25:57.652838 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-298eff13-61ba-46a8-a730-2f3343bce68d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:25:57.666289 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298eff13-61ba-46a8-a730-2f3343bce68d service neutron] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:25:57.671561 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995fa384-a5b5-4adf-a433-cd4c43a1383c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:25:57.672196 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995fa384-a5b5-4adf-a433-cd4c43a1383c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:25:57.672525 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995fa384-a5b5-4adf-a433-cd4c43a1383c admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:25:57.672812 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995fa384-a5b5-4adf-a433-cd4c43a1383c admin admin] PATH_INFO: `/v3/users/8c796e1544354dbe9d1139c77b05f477` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:25:57.673937 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995fa384-a5b5-4adf-a433-cd4c43a1383c admin admin] RBAC: Authorizing `identity:delete_user(user_id=8c796e1544354dbe9d1139c77b05f477)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:25:57.676085 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995fa384-a5b5-4adf-a433-cd4c43a1383c admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:25:57.687541 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:25:57.871503 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-995fa384-a5b5-4adf-a433-cd4c43a1383c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8c796e1544354dbe9d1139c77b05f477'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:25:57.878322 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-995fa384-a5b5-4adf-a433-cd4c43a1383c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8c796e1544354dbe9d1139c77b05f477'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:25:57.878996 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-995fa384-a5b5-4adf-a433-cd4c43a1383c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8c796e1544354dbe9d1139c77b05f477'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:25:57.956382 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-995fa384-a5b5-4adf-a433-cd4c43a1383c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8c796e1544354dbe9d1139c77b05f477'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:25:57.969102 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-995fa384-a5b5-4adf-a433-cd4c43a1383c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8c796e1544354dbe9d1139c77b05f477'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:25:57.975103 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1454/2932] 10.222.0.46 () {58 vars in 1224 bytes} [Mon Oct 10 12:25:57 2022] DELETE /identity/v3/users/8c796e1544354dbe9d1139c77b05f477 => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:25:58.226461 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7a745c-ab1c-4c55-9d39-f8e5d100d509 service placement] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:25:58.238123 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7a745c-ab1c-4c55-9d39-f8e5d100d509 service placement] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:25:58.297205 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5d91d0-0c75-4a69-9e51-e3082f76f3a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:25:58.299291 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5d91d0-0c75-4a69-9e51-e3082f76f3a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:25:58.299291 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5d91d0-0c75-4a69-9e51-e3082f76f3a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:25:58.299291 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5d91d0-0c75-4a69-9e51-e3082f76f3a5 admin admin] PATH_INFO: `/v3/projects/c6ab1776de294ca1a23fc110b2f9af1d` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:25:58.299679 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5d91d0-0c75-4a69-9e51-e3082f76f3a5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c6ab1776de294ca1a23fc110b2f9af1d)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:25:58.301596 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5d91d0-0c75-4a69-9e51-e3082f76f3a5 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:25:58.404213 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:25:58.419971 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-ff5d91d0-0c75-4a69-9e51-e3082f76f3a5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c6ab1776de294ca1a23fc110b2f9af1d', 'request_id': 'req-ff5d91d0-0c75-4a69-9e51-e3082f76f3a5'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:25:58.425362 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-ff5d91d0-0c75-4a69-9e51-e3082f76f3a5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c6ab1776de294ca1a23fc110b2f9af1d', 'request_id': 'req-ff5d91d0-0c75-4a69-9e51-e3082f76f3a5'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:25:58.431560 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-ff5d91d0-0c75-4a69-9e51-e3082f76f3a5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c6ab1776de294ca1a23fc110b2f9af1d', 'request_id': 'req-ff5d91d0-0c75-4a69-9e51-e3082f76f3a5'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:25:58.543053 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-ff5d91d0-0c75-4a69-9e51-e3082f76f3a5 admin admin] The token cache is being invalidate because project c6ab1776de294ca1a23fc110b2f9af1d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81677) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 10 12:25:58.544251 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-ff5d91d0-0c75-4a69-9e51-e3082f76f3a5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:25:58.544463 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-ff5d91d0-0c75-4a69-9e51-e3082f76f3a5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:25:58.545972 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1479/2933] 10.222.0.46 () {58 vars in 1233 bytes} [Mon Oct 10 12:25:58 2022] DELETE /identity/v3/projects/c6ab1776de294ca1a23fc110b2f9af1d => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:25:58.550141 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995fa384-a5b5-4adf-a433-cd4c43a1383c admin admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:25:58.551561 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-995fa384-a5b5-4adf-a433-cd4c43a1383c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:25:58.565417 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995fa384-a5b5-4adf-a433-cd4c43a1383c admin admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:25:58.620859 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9246a3-d3c5-4eb0-80df-37ba551b595e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:25:58.621504 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9246a3-d3c5-4eb0-80df-37ba551b595e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:25:58.622322 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9246a3-d3c5-4eb0-80df-37ba551b595e admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:25:58.623065 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9246a3-d3c5-4eb0-80df-37ba551b595e admin admin] PATH_INFO: `/v3/users/32e9c2c51e664bb6acb8ce1693170385` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:25:58.623742 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9246a3-d3c5-4eb0-80df-37ba551b595e admin admin] RBAC: Authorizing `identity:delete_user(user_id=32e9c2c51e664bb6acb8ce1693170385)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:25:58.625763 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9246a3-d3c5-4eb0-80df-37ba551b595e admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:25:58.815559 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-5a9246a3-d3c5-4eb0-80df-37ba551b595e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '32e9c2c51e664bb6acb8ce1693170385'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:25:58.822845 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-5a9246a3-d3c5-4eb0-80df-37ba551b595e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '32e9c2c51e664bb6acb8ce1693170385'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:25:58.823561 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-5a9246a3-d3c5-4eb0-80df-37ba551b595e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '32e9c2c51e664bb6acb8ce1693170385'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:25:58.921491 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-5a9246a3-d3c5-4eb0-80df-37ba551b595e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '32e9c2c51e664bb6acb8ce1693170385'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:25:58.934369 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-5a9246a3-d3c5-4eb0-80df-37ba551b595e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '32e9c2c51e664bb6acb8ce1693170385'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:25:58.940385 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1455/2934] 10.222.0.46 () {58 vars in 1224 bytes} [Mon Oct 10 12:25:58 2022] DELETE /identity/v3/users/32e9c2c51e664bb6acb8ce1693170385 => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:25:59.180733 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5d91d0-0c75-4a69-9e51-e3082f76f3a5 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:25:59.183723 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-ff5d91d0-0c75-4a69-9e51-e3082f76f3a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:25:59.204074 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5d91d0-0c75-4a69-9e51-e3082f76f3a5 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:25:59.262902 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4050b9c7-cd1a-45c1-9529-101faeaf3e61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:25:59.263572 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4050b9c7-cd1a-45c1-9529-101faeaf3e61 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:25:59.263862 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4050b9c7-cd1a-45c1-9529-101faeaf3e61 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:25:59.264141 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4050b9c7-cd1a-45c1-9529-101faeaf3e61 admin admin] PATH_INFO: `/v3/projects/a198397258e44f2fa172d6b5289124de` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:25:59.265305 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4050b9c7-cd1a-45c1-9529-101faeaf3e61 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a198397258e44f2fa172d6b5289124de)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:25:59.267271 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4050b9c7-cd1a-45c1-9529-101faeaf3e61 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:25:59.373198 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-4050b9c7-cd1a-45c1-9529-101faeaf3e61 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a198397258e44f2fa172d6b5289124de', 'request_id': 'req-4050b9c7-cd1a-45c1-9529-101faeaf3e61'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:25:59.378704 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-4050b9c7-cd1a-45c1-9529-101faeaf3e61 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a198397258e44f2fa172d6b5289124de', 'request_id': 'req-4050b9c7-cd1a-45c1-9529-101faeaf3e61'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:25:59.384947 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-4050b9c7-cd1a-45c1-9529-101faeaf3e61 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a198397258e44f2fa172d6b5289124de', 'request_id': 'req-4050b9c7-cd1a-45c1-9529-101faeaf3e61'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:25:59.481791 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-4050b9c7-cd1a-45c1-9529-101faeaf3e61 admin admin] The token cache is being invalidate because project a198397258e44f2fa172d6b5289124de was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81677) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 10 12:25:59.482084 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-4050b9c7-cd1a-45c1-9529-101faeaf3e61 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:25:59.482792 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-4050b9c7-cd1a-45c1-9529-101faeaf3e61 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:25:59.484299 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1480/2935] 10.222.0.46 () {58 vars in 1233 bytes} [Mon Oct 10 12:25:59 2022] DELETE /identity/v3/projects/a198397258e44f2fa172d6b5289124de => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:26:08.936828 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:DELETE=20,keystone:SELECT=75,keystone:INSERT=2 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:26:09.478449 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=65,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:26:32.737647 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9246a3-d3c5-4eb0-80df-37ba551b595e admin admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:26:32.739304 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-5a9246a3-d3c5-4eb0-80df-37ba551b595e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:26:32.810452 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9246a3-d3c5-4eb0-80df-37ba551b595e admin admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:26:32.870089 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac91161f-fa2d-49ae-919a-f7dc2216f428 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e8f9bb1776d47169ef7febb57c954cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad3772f7b6a47c28b7ba37a9316a3c1', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:26:32.870628 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac91161f-fa2d-49ae-919a-f7dc2216f428 service placement] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:26:32.870941 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac91161f-fa2d-49ae-919a-f7dc2216f428 service placement] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:26:32.871219 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac91161f-fa2d-49ae-919a-f7dc2216f428 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:26:32.873031 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-ac91161f-fa2d-49ae-919a-f7dc2216f428 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:26:32.884455 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac91161f-fa2d-49ae-919a-f7dc2216f428 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:26:32.886025 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac91161f-fa2d-49ae-919a-f7dc2216f428 service placement] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:26:32.896611 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1456/2936] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 10 12:26:32 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 10 12:26:34.533838 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5a496b-14cb-4835-a4be-a69cbbdc7a4c None None] REQUEST_METHOD: `POST` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:26:34.534414 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5a496b-14cb-4835-a4be-a69cbbdc7a4c None None] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:26:34.534414 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5a496b-14cb-4835-a4be-a69cbbdc7a4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:26:34.564948 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.auth.core [None req-8c5a496b-14cb-4835-a4be-a69cbbdc7a4c None None] MFA Rules not processed for user `d6430593eecf45d8b891c3b55de4a944`. Rule list: `[]` (Enabled: `True`). {{(pid=81677) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 10 12:26:34.566308 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-8c5a496b-14cb-4835-a4be-a69cbbdc7a4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:26:34.631550 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1481/2937] 10.222.0.46 () {58 vars in 967 bytes} [Mon Oct 10 12:26:34 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 10 12:26:34.672342 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac91161f-fa2d-49ae-919a-f7dc2216f428 service placement] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:26:34.673700 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-ac91161f-fa2d-49ae-919a-f7dc2216f428 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:26:34.742922 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac91161f-fa2d-49ae-919a-f7dc2216f428 service placement] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:26:34.747862 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c957ee61-8277-42d5-95ef-a749c4e28382 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15ef3d53ea484237adb0b3d64fdc5480', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad3772f7b6a47c28b7ba37a9316a3c1', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:26:34.748468 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c957ee61-8277-42d5-95ef-a749c4e28382 service swift] REQUEST_METHOD: `GET` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:26:34.748751 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c957ee61-8277-42d5-95ef-a749c4e28382 service swift] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:26:34.749027 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c957ee61-8277-42d5-95ef-a749c4e28382 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:26:34.757671 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c957ee61-8277-42d5-95ef-a749c4e28382 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:26:34.759288 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c957ee61-8277-42d5-95ef-a749c4e28382 service swift] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:26:34.767357 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1457/2938] 10.222.0.46 () {60 vars in 1384 bytes} [Mon Oct 10 12:26:34 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 10 12:26:43.368712 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5a496b-14cb-4835-a4be-a69cbbdc7a4c None None] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:26:43.370325 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-8c5a496b-14cb-4835-a4be-a69cbbdc7a4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:26:43.442956 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5a496b-14cb-4835-a4be-a69cbbdc7a4c None None] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:26:43.447552 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32dee58-8369-4b94-9d3d-61a15b5cc184 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:26:43.448194 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32dee58-8369-4b94-9d3d-61a15b5cc184 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:26:43.448478 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32dee58-8369-4b94-9d3d-61a15b5cc184 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:26:43.448760 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32dee58-8369-4b94-9d3d-61a15b5cc184 admin admin] PATH_INFO: `/v3/users/2db334d548cf45a7a338a7fcc30ac3c4` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:26:43.449939 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32dee58-8369-4b94-9d3d-61a15b5cc184 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2db334d548cf45a7a338a7fcc30ac3c4)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:26:43.451915 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32dee58-8369-4b94-9d3d-61a15b5cc184 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:26:43.843600 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-e32dee58-8369-4b94-9d3d-61a15b5cc184 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2db334d548cf45a7a338a7fcc30ac3c4'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:26:43.851273 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-e32dee58-8369-4b94-9d3d-61a15b5cc184 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2db334d548cf45a7a338a7fcc30ac3c4'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:26:43.851974 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-e32dee58-8369-4b94-9d3d-61a15b5cc184 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2db334d548cf45a7a338a7fcc30ac3c4'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:26:43.937914 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-e32dee58-8369-4b94-9d3d-61a15b5cc184 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2db334d548cf45a7a338a7fcc30ac3c4'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:26:43.950780 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-e32dee58-8369-4b94-9d3d-61a15b5cc184 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2db334d548cf45a7a338a7fcc30ac3c4'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:26:43.957525 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1482/2939] 10.222.0.46 () {58 vars in 1224 bytes} [Mon Oct 10 12:26:43 2022] DELETE /identity/v3/users/2db334d548cf45a7a338a7fcc30ac3c4 => generated 0 bytes in 589 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:26:44.248626 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c957ee61-8277-42d5-95ef-a749c4e28382 service swift] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:26:44.268624 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c957ee61-8277-42d5-95ef-a749c4e28382 service swift] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:26:44.290082 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=100 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:26:44.331691 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de68074d-0d64-449b-a038-23e0101436aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:26:44.332895 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de68074d-0d64-449b-a038-23e0101436aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:26:44.333185 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de68074d-0d64-449b-a038-23e0101436aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:26:44.333473 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de68074d-0d64-449b-a038-23e0101436aa admin admin] PATH_INFO: `/v3/projects/bafe96e8ab4d498981d0e309c484c125` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:26:44.334625 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de68074d-0d64-449b-a038-23e0101436aa admin admin] RBAC: Authorizing `identity:delete_project(project_id=bafe96e8ab4d498981d0e309c484c125)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:26:44.336610 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de68074d-0d64-449b-a038-23e0101436aa admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:26:44.481862 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-de68074d-0d64-449b-a038-23e0101436aa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bafe96e8ab4d498981d0e309c484c125', 'request_id': 'req-de68074d-0d64-449b-a038-23e0101436aa'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:26:44.488089 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-de68074d-0d64-449b-a038-23e0101436aa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bafe96e8ab4d498981d0e309c484c125', 'request_id': 'req-de68074d-0d64-449b-a038-23e0101436aa'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:26:44.495359 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-de68074d-0d64-449b-a038-23e0101436aa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bafe96e8ab4d498981d0e309c484c125', 'request_id': 'req-de68074d-0d64-449b-a038-23e0101436aa'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:26:44.595165 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-de68074d-0d64-449b-a038-23e0101436aa admin admin] The token cache is being invalidate because project bafe96e8ab4d498981d0e309c484c125 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81676) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 10 12:26:44.595473 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-de68074d-0d64-449b-a038-23e0101436aa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:26:44.596102 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-de68074d-0d64-449b-a038-23e0101436aa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:26:44.597802 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1458/2940] 10.222.0.46 () {58 vars in 1233 bytes} [Mon Oct 10 12:26:44 2022] DELETE /identity/v3/projects/bafe96e8ab4d498981d0e309c484c125 => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:26:44.602575 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32dee58-8369-4b94-9d3d-61a15b5cc184 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:26:44.604723 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-e32dee58-8369-4b94-9d3d-61a15b5cc184 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:26:44.618773 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32dee58-8369-4b94-9d3d-61a15b5cc184 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:26:44.623809 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:26:44.678316 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15aeb53b-c8c1-4305-94ce-eb4274770344 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:26:44.678991 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15aeb53b-c8c1-4305-94ce-eb4274770344 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:26:44.679331 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15aeb53b-c8c1-4305-94ce-eb4274770344 admin admin] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:26:44.679612 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15aeb53b-c8c1-4305-94ce-eb4274770344 admin admin] PATH_INFO: `/v3/users/aae29306bca048dab6445e5db59a8a95` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:26:44.680761 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15aeb53b-c8c1-4305-94ce-eb4274770344 admin admin] RBAC: Authorizing `identity:delete_user(user_id=aae29306bca048dab6445e5db59a8a95)` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:26:44.682731 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15aeb53b-c8c1-4305-94ce-eb4274770344 admin admin] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:26:44.877803 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-15aeb53b-c8c1-4305-94ce-eb4274770344 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aae29306bca048dab6445e5db59a8a95'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:26:44.886020 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-15aeb53b-c8c1-4305-94ce-eb4274770344 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aae29306bca048dab6445e5db59a8a95'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:26:44.886758 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-15aeb53b-c8c1-4305-94ce-eb4274770344 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aae29306bca048dab6445e5db59a8a95'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:26:44.967389 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-15aeb53b-c8c1-4305-94ce-eb4274770344 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aae29306bca048dab6445e5db59a8a95'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:26:44.981176 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.notifications [None req-15aeb53b-c8c1-4305-94ce-eb4274770344 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aae29306bca048dab6445e5db59a8a95'} {{(pid=81677) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:26:44.987772 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1483/2941] 10.222.0.46 () {58 vars in 1224 bytes} [Mon Oct 10 12:26:44 2022] DELETE /identity/v3/users/aae29306bca048dab6445e5db59a8a95 => generated 0 bytes in 386 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 10 12:26:45.239349 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de68074d-0d64-449b-a038-23e0101436aa admin admin] Authenticating user token {{(pid=81676) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:26:45.239349 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.fernet_utils [None req-de68074d-0d64-449b-a038-23e0101436aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81676) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:26:45.248674 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de68074d-0d64-449b-a038-23e0101436aa admin admin] Validating token access rules against request {{(pid=81676) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:26:45.315369 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6867e75c-c598-4ea3-8959-76683ff52ba5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8cbbc9233ced43d487079e9479dc555c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec187f354d154dc98e045921aa4ca7fd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81676) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:26:45.316021 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6867e75c-c598-4ea3-8959-76683ff52ba5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:26:45.316306 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6867e75c-c598-4ea3-8959-76683ff52ba5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:26:45.316583 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6867e75c-c598-4ea3-8959-76683ff52ba5 admin admin] PATH_INFO: `/v3/projects/91f85d93de9b44dc9ef044e3f7ecb7c4` {{(pid=81676) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:26:45.317728 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6867e75c-c598-4ea3-8959-76683ff52ba5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=91f85d93de9b44dc9ef044e3f7ecb7c4)` {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:26:45.319807 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6867e75c-c598-4ea3-8959-76683ff52ba5 admin admin] RBAC: Authorization granted {{(pid=81676) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:26:45.471418 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-6867e75c-c598-4ea3-8959-76683ff52ba5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '91f85d93de9b44dc9ef044e3f7ecb7c4', 'request_id': 'req-6867e75c-c598-4ea3-8959-76683ff52ba5'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:26:45.476047 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:26:45.478334 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-6867e75c-c598-4ea3-8959-76683ff52ba5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '91f85d93de9b44dc9ef044e3f7ecb7c4', 'request_id': 'req-6867e75c-c598-4ea3-8959-76683ff52ba5'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:26:45.485042 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-6867e75c-c598-4ea3-8959-76683ff52ba5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '91f85d93de9b44dc9ef044e3f7ecb7c4', 'request_id': 'req-6867e75c-c598-4ea3-8959-76683ff52ba5'} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:26:45.595049 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-6867e75c-c598-4ea3-8959-76683ff52ba5 admin admin] The token cache is being invalidate because project 91f85d93de9b44dc9ef044e3f7ecb7c4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81676) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 10 12:26:45.595338 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-6867e75c-c598-4ea3-8959-76683ff52ba5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:26:45.595975 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG keystone.notifications [None req-6867e75c-c598-4ea3-8959-76683ff52ba5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81676) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 10 12:26:45.597556 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: [pid: 81676|app: 0|req: 1459/2942] 10.222.0.46 () {58 vars in 1233 bytes} [Mon Oct 10 12:26:45 2022] DELETE /identity/v3/projects/91f85d93de9b44dc9ef044e3f7ecb7c4 => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 10 12:26:54.983868 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG dbcounter [-] [81677] Writing DB stats keystone:SELECT=49,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81677) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:26:55.732187 dv-cinder-860209-4-iscsi devstack@keystone.service[81676]: DEBUG dbcounter [-] [81676] Writing DB stats keystone:DELETE=2,keystone:SELECT=5,keystone:INSERT=1 {{(pid=81676) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 10 12:32:09.870864 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15aeb53b-c8c1-4305-94ce-eb4274770344 admin admin] Authenticating user token {{(pid=81677) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 10 12:32:09.872378 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-15aeb53b-c8c1-4305-94ce-eb4274770344 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:32:09.942081 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15aeb53b-c8c1-4305-94ce-eb4274770344 admin admin] Validating token access rules against request {{(pid=81677) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 10 12:32:10.001846 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4d5a64-ac91-4e09-98be-befd50c4517d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e8f9bb1776d47169ef7febb57c954cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4ad3772f7b6a47c28b7ba37a9316a3c1', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81677) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 10 12:32:10.002503 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4d5a64-ac91-4e09-98be-befd50c4517d service placement] REQUEST_METHOD: `GET` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 10 12:32:10.002806 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4d5a64-ac91-4e09-98be-befd50c4517d service placement] SCRIPT_NAME: `/identity` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 10 12:32:10.003082 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4d5a64-ac91-4e09-98be-befd50c4517d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81677) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 10 12:32:10.004814 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.fernet_utils [None req-9d4d5a64-ac91-4e09-98be-befd50c4517d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81677) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 10 12:32:10.016221 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4d5a64-ac91-4e09-98be-befd50c4517d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 10 12:32:10.017787 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4d5a64-ac91-4e09-98be-befd50c4517d service placement] RBAC: Authorization granted {{(pid=81677) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 10 12:32:10.027656 dv-cinder-860209-4-iscsi devstack@keystone.service[81677]: [pid: 81677|app: 0|req: 1484/2943] 10.222.0.46 () {60 vars in 1365 bytes} [Mon Oct 10 12:32:09 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0)